{"id":7,"date":"2019-08-26T07:59:48","date_gmt":"2019-08-26T07:59:48","guid":{"rendered":"http:\/\/blogs.igalia.com\/mshin\/?p=7"},"modified":"2019-08-26T07:59:48","modified_gmt":"2019-08-26T07:59:48","slug":"why-should-we-use-a-precise-width-integer-type","status":"publish","type":"post","link":"https:\/\/blogs.igalia.com\/mshin\/2019\/08\/26\/why-should-we-use-a-precise-width-integer-type\/","title":{"rendered":"Why should we use a precise-width integer type?"},"content":{"rendered":"

Introduction<\/strong>
\nA few months ago, I’ve finished the task of replacing imprecise-width integer types like (unsigned) short, int, long, long long<\/em> by precise-width integer types like (u)int16\/int32\/int64_t<\/em> in Blink.. I’ve been working on the Onion Soup project<\/a> and it was my first task of this project. When I took this task, I thought it would be comparatively simple and quite mechanical work, but this was not the case. It was sometimes required to understand whole code flow in-depth even out of blink or to explain the benefit of such changes to reviewers.<\/p>\n

I’d like to introduce why we should prefer using the precise-width integer type and what I did for this task.<\/p>\n

Why should we use the precise-width integer type?<\/strong>
\nWe should keep in mind the fact that the c++ standards don\u2019t specify the specific size of each integer type.<\/p>\n

Data Model Table<\/p>\n\n\n\n\n\n\n\n\n
Data model<\/th>\nsizeof(int)<\/th>\nsizeof(long)<\/th>\nsizeof(long long)<\/th>\nexample<\/th>\n<\/tr>\n<\/thead>\n
LP32<\/td>\n16b<\/td>\n32b<\/td>\n64b<\/td>\nWin16<\/td>\n<\/tr>\n
ILP32<\/td>\n32b<\/td>\n32b<\/td>\n64b<\/td>\nWin32, i386 OSX & Linux<\/td>\n<\/tr>\n
LP64<\/td>\n32b<\/td>\n64b<\/td>\n64b<\/td>\nx86-64 OSX & Linux<\/td>\n<\/tr>\n
LLP64<\/td>\n32b<\/td>\n32b<\/td>\n64b<\/td>\nWin64<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Chromium supports various platforms with one repository and shows definitely the variable of different sizes with (unsigned) long<\/em> between Android-Kitkat\/MacOS\/Win7 buildbots and other buildbots for try-bots. It means that it would have potential security issues like the stack overflow if we treat that long<\/em> has 64 bits precise since the size of long<\/em> is 64 bits like long long<\/em> on almost all of build systems. And we should avoid mixing the use of the precise-width integer type and the imprecise-width integer type to store the same value.<\/p>\n

Actually, after working on this I got a report mentioning that my changes to move from imprecise-width to precise-width types fixed a security issue. This made me realize that more existing issues could still be fixed this way, and I started fixing other ones reported via ClusterFuzz after that.<\/p>\n

Google also recommends this in its own coding styleguide, see here<\/a>.<\/p>\n

Summary of my contributions in Blink<\/strong>
\n1) Decided the order of the changing type.<\/p>\n

\nunsigned short -> uint16_t \/ short -> int16_t\nunsigned long long -> uint64_t \/ long long -> int64_t\nunsigned long -> uint32_t or uint64_t, long -> int32_t or int64_t<\/pre>\n
2) Wrote the patch including only related codes.<\/p>\n
3)  Found what is the proper type.
\nWhich type is proper between uint32_t<\/em> and uint64_t<\/em> when I should change unsigned long<\/em>?
\nEvery time I asked myself the question since I had to consider the fact that unsigned long<\/em> was 32 bit on Mac, Android-kitkat and Window7 buildbot and was 64 bit on others so far. So I needed to understand its code flow and the use cases not only in Blink but also out of Blink. In general, it would be best to avoid integer-width conversions where possible and to only convert to larger types when needed.<\/p>\n
4) Utilized the template instead of the functions for each type
\nI reduced the code quantity with the template and avoided the build error by duplicated definition.<\/p>\n
5) Used base::Optional<\/em> when the initial value is ambiguous.
\nIn general the variable’s initial value would be 0<\/em> or -1<\/em>, but base::Optional<\/em> is more readable and simple than comparing 0<\/em> or -1<\/em> to check the initial value. Actually, I needed to reach a consensus with the review since it could change the code logic.<\/p>\n
bool HasValidSnapshotMetadata() const { return snapshot_size_ >= 0; }\nlong long snapshot_size_;\n<\/code><\/pre>\n
Instead,<\/p>\n
bool HasValidSnapshotMetadata() const { return snapshot_size_.has_value(); }\nbase::Optional<uint64_t> snapshot_size_;\n<\/code><\/pre>\n
6) Used CheckedNumeric<\/em> to check the overflow of the variable.<\/p>\n
static bool SizeCalculationMayOverflow(unsigned width,\n                                         unsigned height,\n                                         unsigned decoded_bytes_per_pixel) {\n    unsigned long long total_size = static_cast<unsigned long long>(width) *\n                                    static_cast<unsigned long long>(height);\n    if (decoded_bytes_per_pixel == 4)\n      return total_size > ((1 << 29) - 1);\n    return total_size > ((1 << 28) - 1);\n}\n<\/code><\/pre>\n
The above code was the old one before I changed and it was for checking if the variable has the overflow, I took some time to fully understand it.
\nI’ve used CheckedNumeric<\/em> since the reviewer let me know this utility.  As you can see from the code below,  it has been changed to be simple and readable.<\/p>\n
  inline bool SizeCalculationMayOverflow(unsigned width,\n                                         unsigned height,\n                                         unsigned decoded_bytes_per_pixel) {\n    base::CheckedNumeric<int32_t> total_size = width;\n    total_size *= height;\n    total_size *= decoded_bytes_per_pixel;\n    return !total_size.IsValid();\n  }\n<\/code><\/pre>\n
7) Oops! MSVC!
\nThe below code shows the different results due to compilers.<\/p>\n
class Foo {\n    uint16_t a;\n    unsigned b : 16;\n};\n<\/code><\/pre>\n
std::cout << sizeof(Foo);\n<\/code><\/pre>\n
The results are from clang<\/strong> : 4 bytes, gcc<\/strong> : 4 bytes and msvc<\/strong> : 8 bytes<\/p>\n
So, it caused the build break with MSVC toolchain on Windows buildbots as the class size was increased. It took a long time to analyze the cause and solve this issue.<\/p>\n
In conclusion, it was very simple to fix. See the code below.<\/p>\n
class Foo {\n    unsigned a : 16;\n    unsigned b : 16;\n};\n<\/code><\/pre>\n
8) Brought out the potential issue with  with the overflow.
\nBlink can handle the width and height within 4 bytes range. The old code showed the abnormal layout result with overflowed width\/height but it did not crash. I added the assertion code to check if there is the overflow when changing the type. Finally, we prevent the overflow and solve to layout the object’s position and size within the maximum value of 4 bytes.<\/p>\n
9) Added PRESUBMIT check.
\nNow that we cleaned up the Blink code from the imprecise-width types, a PRESUBMIT hook was added to prevent them from being reintroduced to the sourcebase. So I added the PRESUBMIT check to ensure new uses do not come in again.<\/p>\n
10) Why do we still use int<\/em> and unsigned<\/em>?
\nThis is the comment of the reviewer and I also agree with it.<\/p>\n
\n  Thee C++ style guide<\/a> states "We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t<\/em> or uint64_t<\/em>."
\n  The style guide does state not to use the imprecise-width integers like long long\/long\/short, but int is the imprecise-width type that is explicitly preferred. Among other reasons, this is likely for readability and the fact that ints are by default the "native"\/fastest type,  and potentially large concerns over code churn. More there is not any benefit to replace int to int32t<\/em> and potentially large concerns over code churn.\n<\/p><\/blockquote>\n
I’m happy to contribute to Chromium & Blink with more stable code through changing to the precise-width integer type and to learn more about this particular domain.<\/p>\n
References<\/strong>
\n[1] https:\/\/en.cppreference.com\/w\/cpp\/language\/types
\n[2] http:\/\/nickdesaulniers.github.io\/blog\/2016\/05\/30\/data-models-and-word-size\/<\/p>\n","protected":false},"excerpt":{"rendered":"
Introduction A few months ago, I’ve finished the task of replacing imprecise-width integer types like (unsigned) short, int, long, long long by precise-width integer types like (u)int16\/int32\/int64_t in Blink.. I’ve been working on the Onion Soup project and it was my first task of this project. When I took this task, I thought it would … Continue reading Why should we use a precise-width integer type?<\/span><\/a><\/p>\n","protected":false},"author":58,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,3],"tags":[5,8,4,6,7],"class_list":["post-7","post","type-post","status-publish","format-standard","hentry","category-chromium","category-igalia","tag-blink","tag-c","tag-chromium","tag-igalia","tag-precise-width-integer"],"_links":{"self":[{"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/posts\/7","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/users\/58"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/comments?post=7"}],"version-history":[{"count":40,"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/posts\/7\/revisions"}],"predecessor-version":[{"id":56,"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/posts\/7\/revisions\/56"}],"wp:attachment":[{"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/media?parent=7"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/categories?post=7"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.igalia.com\/mshin\/wp-json\/wp\/v2\/tags?post=7"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}