{"id":566,"date":"2015-08-14T12:22:29","date_gmt":"2015-08-14T10:22:29","guid":{"rendered":"http:\/\/blogs.igalia.com\/berto\/?p=566"},"modified":"2017-01-25T17:22:35","modified_gmt":"2017-01-25T16:22:35","slug":"io-limits-for-disk-groups-in-qemu-2-4","status":"publish","type":"post","link":"https:\/\/blogs.igalia.com\/berto\/2015\/08\/14\/io-limits-for-disk-groups-in-qemu-2-4\/","title":{"rendered":"I\/O limits for disk groups in QEMU 2.4"},"content":{"rendered":"<p>QEMU <a href=\"http:\/\/wiki.qemu.org\/ChangeLog\/2.4\">2.4.0<\/a> has just been released, and among many other things it comes with some of the stuff I have been working on lately. In this blog post I am going to talk about disk I\/O limits and the new feature to group several disks together.<\/p>\n<h2>Disk I\/O limits<\/h2>\n<p>Disk I\/O limits allow us to control the amount of I\/O that a guest can perform. This is useful for example if we have several VMs in the same host and we want to reduce the impact they have on each other if the disk usage is very high.<\/p>\n<p>The I\/O limits can be set using the QMP command <tt>block_set_io_throttle<\/tt>, or with the command line using the <tt>throttling.*<\/tt> options for the <tt>-drive<\/tt> parameter (in brackets in the examples below). Both the throughput and the number of I\/O operations can be limited. For a more fine-grained control, the limits of each one of them can be set on read operations, write operations, or the combination of both:<\/p>\n<ul>\n<li><em>bps<\/em> (<em>throttling.bps-total<\/em>): Total throughput limit (in bytes\/second).<\/li>\n<li><em>bps_rd<\/em> (<em>throttling.bps-read<\/em>): Read throughput limit.<\/li>\n<li><em>bps_wr<\/em> (<em>throttling.bps-write<\/em>): Write throughput limit.<\/li>\n<li><em>iops<\/em> (<em>throttling.iops-total<\/em>): Total I\/O operations per second.<\/li>\n<li><em>iops_rd<\/em> (<em>throttling.iops-read<\/em>): Read I\/O operations per second.<\/li>\n<li><em>iops_wr<\/em> (<em>throttling.iops-write<\/em>): Write I\/O operations per second.<\/li>\n<\/ul>\n<p>Example:<\/p>\n<pre>\r\n-drive if=virtio,file=hd1.qcow2,throttling.bps-write=52428800,throttling.iops-total=6000\r\n<\/pre>\n<p>In addition to that, it is also possible to configure the maximum burst size, which defines a pool of I\/O that the guest can perform without being limited:<\/p>\n<ul>\n<li><em>bps_max<\/em> (<em>throttling.bps-total-max<\/em>): Total maximum (in bytes).<\/li>\n<li><em>bps_rd_max<\/em> (<em>throttling.bps-read-max<\/em>): Read maximum.<\/li>\n<li><em>bps_wr_max<\/em> (<em>throttling.bps-write-max<\/em>): Write maximum.<\/li>\n<li><em>iops_max<\/em> (<em>throttling.iops-total-max<\/em>): Total maximum of I\/O operations.<\/li>\n<li><em>iops_rd_max<\/em> (<em>throttling.iops-read-max<\/em>): Read I\/O operations.<\/li>\n<li><em>iops_wr_max<\/em> (<em>throttling.iops-write-max<\/em>): Write I\/O operations.<\/li>\n<\/ul>\n<p>One additional parameter named <tt>iops_size<\/tt> allows us to deal with the case where big I\/O operations can be used to bypass the limits we have set. In this case, if a particular I\/O operation is bigger than <tt>iops_size<\/tt> then it is counted several times when it comes to calculating the I\/O limits. So a 128KB request will be counted as 4 requests if <tt>iops_size<\/tt> is 32KB.<\/p>\n<ul>\n<li><em>iops_size<\/em> (<em>throttling.iops-size<\/em>): Size of an I\/O request (in bytes).<\/li>\n<\/ul>\n<h2>Group throttling<\/h2>\n<p>All of these parameters I&#8217;ve just described operate on individual disk drives and have been available for a while. Since QEMU 2.4 however, it is also possible to have several drives share the same limits. This is configured using the new <tt>group<\/tt> parameter.<\/p>\n<p>The way it works is that each disk with I\/O limits is member of a <em>throttle group<\/em>, and the limits apply to the combined I\/O of all group members using a round-robin algorithm. The way to put several disks together is just to use the <tt>group<\/tt> parameter with all of them using the same group name. Once the group is set, there&#8217;s no need to pass the parameter to <tt>block_set_io_throttle<\/tt> anymore unless we want to move the drive to a different group. Since the I\/O limits apply to all group members, it is enough to use <tt>block_set_io_throttle<\/tt> in just one of them.<\/p>\n<p>Here&#8217;s an example of how to set groups using the command line:<\/p>\n<pre>\r\n-drive if=virtio,file=hd1.qcow2,throttling.iops-total=6000,throttling.group=foo\r\n-drive if=virtio,file=hd2.qcow2,throttling.iops-total=6000,throttling.group=foo\r\n-drive if=virtio,file=hd3.qcow2,throttling.iops-total=3000,throttling.group=bar\r\n-drive if=virtio,file=hd4.qcow2,throttling.iops-total=6000,throttling.group=foo\r\n-drive if=virtio,file=hd5.qcow2,throttling.iops-total=3000,throttling.group=bar\r\n-drive if=virtio,file=hd6.qcow2,throttling.iops-total=5000\r\n<\/pre>\n<p>In this example, hd1, hd2 and hd4 are all members of a group named <em>foo<\/em> with a combined IOPS limit of 6000, and hd3 and hd5 are members of <em>bar<\/em>. hd6 is left alone (technically it is part of a 1-member group).<\/p>\n<h2>Next steps<\/h2>\n<p>I am currently working on providing more I\/O statistics for disk drives, including latencies and average queue depth on a user-defined interval. The code is almost ready. Next week I will be in Seattle for the <a href=\"http:\/\/events.linuxfoundation.org\/events\/kvm-forum\/\">KVM Forum<\/a> where I will hopefully be able to finish the remaining bits.<\/p>\n<table align=\"center\">\n<tr>\n<td><img decoding=\"async\" src=\"\/berto\/files\/2015\/08\/kvmforum.png\" \/><\/td>\n<\/tr>\n<\/table>\n<p>I will also attend <a href=\"http:\/\/events.linuxfoundation.org\/events\/linuxcon-north-america\">LinuxCon North America<\/a>. <a href=\"http:\/\/www.igalia.com\/\">Igalia<\/a> is sponsoring the event and we have a booth there. Come if you want to talk to us or see our latest demos with <a href=\"http:\/\/www.igalia.com\/webkitforwayland\/\">WebKit for Wayland<\/a>.<\/p>\n<p>See you in Seattle!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>QEMU 2.4.0 has just been released, and among many other things it comes with some of the stuff I have been working on lately. In this blog post I am going to talk about disk I\/O limits and the new feature to group several disks together. Disk I\/O limits Disk I\/O limits allow us to [&hellip;]<\/p>\n","protected":false},"author":8,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,3,8,9,10,12,20],"tags":[],"class_list":["post-566","post","type-post","status-publish","format-standard","hentry","category-debian","category-english","category-gnome","category-gpul","category-igalia","category-free-software","category-virt"],"_links":{"self":[{"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/posts\/566","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/comments?post=566"}],"version-history":[{"count":12,"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/posts\/566\/revisions"}],"predecessor-version":[{"id":587,"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/posts\/566\/revisions\/587"}],"wp:attachment":[{"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/media?parent=566"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/categories?post=566"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.igalia.com\/berto\/wp-json\/wp\/v2\/tags?post=566"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}