{"id":28,"date":"2021-10-02T13:07:59","date_gmt":"2021-10-02T13:07:59","guid":{"rendered":"http:\/\/blogs.igalia.com\/aboya\/?p=28"},"modified":"2021-11-29T07:48:27","modified_gmt":"2021-11-29T07:48:27","slug":"setting-up-visualstudio-code-to-work-with-webkitgtk-using-clangd","status":"publish","type":"post","link":"https:\/\/blogs.igalia.com\/aboya\/2021\/10\/02\/setting-up-visualstudio-code-to-work-with-webkitgtk-using-clangd\/","title":{"rendered":"Setting up VisualStudio code to work with WebKitGTK using clangd"},"content":{"rendered":"

Lately I\u2019m working on a refactor in the append pipeline of the MediaSource Extensions implementation of the WebKit for the GStreamer ports. Working on refactors often triggers many build issues, not only because they often encompass a lot of code, but also because it\u2019s very easy to miss errors in the client code when updating an interface.<\/p>\n

The traditional way to tackle this problem is by doing many build cycles: compile, fix the topmost error, and maybe some other errors on view that seem legit (note in C++ it\u2019s very common to have chain errors that are consequence of previous errors), repeat until it builds successfully.<\/p>\n

This approach is not very pleasant in a project like WebKit where an incremental build of a single file takes just enough time to cause the need for a distraction. It\u2019s also worsened when it\u2019s not just one file, but a complete build that may stop at any time, depending on the order the build system chooses for the files. Often it does take more time to wait for the compiler to show the error than to fix the error.<\/p>\n

Unpleasant unfavors motivation, and lack of motivation unfavors productivity, and by the end of the day you are tired and still undone. Somehow it feels like the time spent fixing trivial build issues is substancially more than the time of a build cycle times the number of errors. Whether that perception is accurate or not, I am acutely aware of the huge impact having helpful tooling has on both productivity and quality of life, both while and after you\u2019re done with the work, so I decided to have a look at the state of modern C++ language servers when working on a large codebase like WebKit. Previous experiences were very unsuccessful, but there are people dedicated to this and progress has been made.<\/p>\n

Creating a WebKit project in VS Code<\/h3>\n
    \n
  1. Open the directory containing the WebKit checkout in VS Code.<\/li>\n
  2. WebKit has A LOT of files. If you use Linux you will see a warning telling you increase the number of inotify watchers. Do so if you haven\u2019t done it before, but even then, it will not be enough, because WebKit has more files than the maximum number of inotify watchers supported by the kernel. Also, they use memory.<\/li>\n
  3. Go to File\/Preferences\/Settings, click the Workspace<\/em> tab, search for Files: Watcher Exclude<\/em><\/strong> and add the following patterns:\n
    **\/CMakeFiles\/**\n**\/JSTests\/**\n**\/LayoutTests\/**\n**\/Tools\/buildstream\/cache\/**\n**\/Tools\/buildstream\/repo\/**\n**\/WebKitBuild\/UserFlatpak\/repo\/**<\/code><\/pre>\n
    This will keep the number of watches on a workable 258k. Still a lot, but under the 1M limit.<\/li>\n<\/ol>\n
    How to set up clangd<\/h3>\n
    The following instructions assume you\u2019re using WebKitGTK with the WebKit Flatpak SDK. They should also work for WPE with minimal substitutions.<\/p>\n
      \n
    1. Microsoft has its own C++ plugin for VS Code, which may be installed by default. The authors of the clangd plugin recommend to uninstall the built-in C++ plugin<\/strong>, as running both doesn\u2019t make much sense and could cause conflicts.<\/li>\n
    2. Install the clangd extension for VS Code from the VS Code Marketplace<\/a>.<\/li>\n
    3. The WebKit flatpak SDK already includes clangd, so it\u2019s not necessary to install it if you\u2019re using it. On the other hand, because the flatpak has a virtual filesystem, it\u2019s necessary to map paths from the flatpak to the outside. You can create this wrapper script for this purpose. Make sure to give it execution rights<\/strong> (chmod +x<\/code>).\n
      \n
      #!\/bin\/bash<\/span><\/span>\nset<\/span> -eu<\/span><\/span>\n# https:\/\/stackoverflow.com\/a\/17841619<\/span><\/span>\nfunction<\/span> join_by<\/span> {<\/span> local<\/span> d=${1-}<\/span> f=${2-}<\/span>; if<\/span> shift<\/span> 2;<\/span> then<\/span> printf<\/span> %s \"<\/span>$f<\/span>\"<\/span> \"<\/span>${@\/<\/span>#\/$d}<\/span>\"<\/span>;<\/span> fi<\/span>; }<\/span><\/span>\n<\/span>\nlocal_webkit=<\/span>\/webkit<\/span>\ninclude_path=(<\/span>\"<\/span>$local_webkit<\/span>\"<\/span>\/WebKitBuild\/UserFlatpak\/runtime\/org.webkit.Sdk\/x86_64\/*\/active\/files\/include)<\/span><\/span>\nif<\/span> [<\/span> !<\/span> -f<\/span> \"<\/span>${include_path[0]}<\/span>\/stdio.h\"<\/span> ]<\/span>; then<\/span><\/span>\n  echo<\/span> \"Couldn't find the directory hosting the \/usr\/include of the flatpak SDK.\"<\/span><\/span>\n  exit<\/span> 1<\/span>\nfi<\/span><\/span>\ninclude_path=<\/span>\"<\/span>${include_path[0]}<\/span>\"<\/span><\/span>\nmappings=(<\/span><\/span>\n  \"<\/span>$local_webkit<\/span>\/WebKitBuild\/GTK\/Debug=\/app\/webkit\/WebKitBuild\/Debug\"<\/span><\/span>\n  \"<\/span>$local_webkit<\/span>\/WebKitBuild\/GTK\/Release=\/app\/webkit\/WebKitBuild\/Release\"<\/span><\/span>\n  \"<\/span>$local_webkit<\/span>=\/app\/webkit\"<\/span><\/span>\n  \"<\/span>$include_path<\/span>=\/usr\/include\"<\/span><\/span>\n)<\/span>\n<\/span>\nexec<\/span> \"<\/span>$local_webkit<\/span>\"<\/span>\/Tools\/Scripts\/webkit-flatpak --gtk --debug run -c clangd --path-mappings=\"<\/span>$(<\/span>join_by<\/span> , \"<\/span>${mappings[@]}<\/span>\"<\/span>)<\/span>\"<\/span> \"<\/span>$@<\/span>\"<\/span><\/span><\/code><\/pre>\n<\/div>\n
      Make sure to set the path of your WebKit repository in local_webkit<\/code>.<\/p>\n
      Then, in VS Code, go to File\/Preferences\/Settings, and in the left pane, search for Extensions\/clangd<\/strong>. Change Clangd: Path<\/em><\/strong> to the absolute path of the saved script above. I recomend making these changes in the Workspace tab, so they apply only to WebKit.<\/strong><\/li>\n
    4. Create a symlink named compile_commands.json<\/code> inside the root of the WebKit checkout directory pointing to the compile_commands.json<\/code> file of the WebKit build you will be using, for instance: WebKitBuild\/GTK\/Debug\/compile_commands.json<\/code><\/li>\n
    5. Create a .clangd<\/code> file inside the root of the WebKit checkout directory with these contents:\n
      If:\n    PathMatch: \"(\/app\/webkit\/)?Source\/.*\\\\.h\"\n    PathExclude: \"(\/app\/webkit\/)?Source\/ThirdParty\/.*\"\n\nCompileFlags:\n    Add: [-include, config.h]<\/code><\/pre>\n
      This includes config.h<\/code> in header files in WebKit files, with the exception of those in Source\/ThirdParty<\/code>. Note: If you need to add additional rules, this is done by adding additional YAML documents<\/em>, which are separated by a ---<\/code> line.<\/li>\n
    6. VS Code clangd plugin doesn\u2019t read .clangd<\/code> by default. Instead, it has to be instructed to do so by adding --enable-config<\/code> to Clangd: Arguments<\/em><\/strong>. Also add --limit-results=5000<\/code>, since the default limit for cross reference search results (100) is too small for WebKit.Additional tip<\/strong>: clangd will also add #include<\/code> lines when you autocomplete a type. While the intention is good, this often can lead to spurious redundant includes. I have disabled it by adding --header-insertion=never<\/code> to clangd\u2019s arguments.<\/li>\n
    7. Restart VS Code. Next time you open a C++ file you will get a prompt requesting confirmating your edited configuration:\n
      <\/div>\n<\/li>\n<\/ol>\n
      VS Code will start indexing your code, and you will see a progress count in the status bar.<\/p>\n
      Debugging problems<\/h3>\n
      clangd has a log. To see it, click View\/Output<\/em><\/strong>, then in the Output panel combo box, select clangd<\/em>.<\/p>\n
      <\/div>\n
      The clangd database is stored in .cache\/clangd<\/code> inside the WebKit checkout directory. rm -rf<\/code>\u2019ing that directory will reset it back to its initial state.<\/p>\n
      For each compilation unit indexed, you\u2019ll find a file following the pattern .cache\/clangd\/index\/<Name>.<Hash>.idx<\/code>. For instance: .cache\/clangd\/index\/MediaSampleGStreamer.cpp.0E0C77DCC76C3567.idx<\/code>. This way you can check whether a particular compilation unit has been indexed.<\/p>\n
      Bug: Some files are not indexed<\/h3>\n
      You may notice VS Code has not indexed all your files. This is apparent when using the Find all references<\/em> feature, since you may be missing results. This in particular affects to generated code, in particular unified sources (.cpp files generated by concatenating via #include<\/code> a series of related .cpp files with the purpose of speeding up the build, compared to compiling them as individual units).<\/p>\n
      I don\u2019t know the reason for this bug, but I can confirm the following workaround: Open a UnifiedSources file. Any UnifiedSources file will do. You can find them in paths such as WebKitBuild\/GTK\/Debug\/WebCore\/DerivedSources\/unified-sources\/UnifiedSource-043dd90b-1.cpp<\/code>. After you open any of them, you\u2019ll see VS Code indexing over a thousand files that were skipped before. You can close the file now. Find all references<\/em> should work once the indexing is done.<\/p>\n
      Things that work<\/h3>\n
      Overall I\u2019m quite satisfied with the setup. The following features work:<\/p>\n