![\"Within](\"https://blogs.igalia.com/dape/2026/03/26/the-implementation-of-container-timing-aggregating-paints-in-blink/images/propagation-to-ancestor.png\")
Measuring paint performance is a balancing act: you need precision, but the measurement itself can’t slow things down.
\nIn my previous post, I introduced Container Timing, a new web API allowing developers to measure the rendering performance of DOM subtrees. Today, I will dive into the technical details of how I implemented this in Blink, the rendering engine used by Chromium.
\nIn Blink, the rendering pipeline goes through several stages: Style, Layout, Paint, and Composite. The Container Timing implementation relies heavily on the Paint stage.
\nThe main idea was not reinventing the wheel. Blink already provides paint timing detection for the implementation of Large Contentful Paint (LCP) and Element Timing. However, this is targeted for specific nodes (an image, a text block). In Container Timing we care about subtrees.
\nSo, when a paint is detected, we need to quickly decide whether the paint is relevant to Container Timing.
\nAs the DOM tree is built (on parsing, or because of a script), we check the value of the attribute containertiming for each Element. When found, we flag that element and all its descendants with the flag SelfOrAncestorHasContainerTiming.
We also have the attribute containertiming-ignore. When found, we will stop the propagation.
So, later, for any paint, we will immediately know if the paint should be tracked for Container Timing or not. This minimizes the impact when the element is not tracked.
\nWhat about DOM tree updates after parsing?
This is a pain point for performance. When a DOM element starts/stops having the containertiming or containertiming-ignore attribute after the DOM tree is created, we need to traverse the tree to update the flag.
When a paint is detected, we just reuse the existing implementation in the ImagePaintTimingDetector and TextPaintTimingDetector, that are also used for LCP and Element Timing for the relevant elements.
Note
Only text and image paints are currently tracked. Video, canvas, and SVG are not yet supported.
\nWe first determine if the paint should be recorded for Container Timing. And this is fast because of the SelfOrAncestorHasContainerTiming flag.
The timing detectors give us the area of the visual rectangle, the bounding box on screen that was painted.
\nFor Container Timing, we added a mechanism to walk up the DOM tree from the painted node. If we encounter an ancestor that is marked with the containertiming attribute (a container timing root), we report that paint event to it.
This “bubbling up” of paint events is illustrated in the diagram below.
\n
Is this expensive?
It depends on the depth of the hierarchy from the node to the most remote ancestor. Further work will be needed to speed up or avoid these traversals.
\nOne of the most interesting challenges was determining the size of the container. It is not just the size of the container timing root. It is the union of all painted content.
Two reasons for this:
\nWe maintain a PaintedRegion for each container. This is a non-overlapping union of the rectangles that cover the updated area:
CurrentRegion = Union(CurrentRegion, NewPaintRect).So, as paints are detected, each container will aggregate the parts of the screen that have been painted by all their children.
\nWe use cc::Region, based on SkRegion from the Skia graphics library to handle these unions efficiently.
The following diagram shows this process in action over three frames.
\n![\"The](\"https://blogs.igalia.com/dape/2026/03/26/the-implementation-of-container-timing-aggregating-paints-in-blink/images/painted-regions.png\")
Because a container paints over multiple frames (e.g., text renders first, then a background image, then a lazy-loaded icon), we cannot just emit one entry. We generate candidates.
\nFor each container, when a paint that increases the painted region is detected, we schedule a new event. Right at the end of the frame presentation, we package the current state into a new performance timeline entry: a PerformanceContainerTiming object.
This object contains:
\nstartTime: The presentation time of the paint. In the Chromium implementation, this is set to the moment the frame was presented to the user, and matches presentationTime from PaintTimingMixin.firstRenderTime: the time of the first paint we detected in the container. Useful for getting a hint of how long a component has been showing updates to the user.identifier is the value of the containertiming attribute. rootElement is the actual element.size: The total area of the aggregated PaintedRegion.lastPaintedElement: the last element that triggered a paint — handy for debugging which child caused the latest candidate.Note
We support the PaintTimingMixin, which adds paintTime (when the paint was committed to the compositor) and presentationTime (when the frame was presented to the user). In Chromium, startTime is set to presentationTime.
This design means the observer might receive multiple entries for the same container. This is intentional: it lets developers pick the milestone that matters to them, typically the point where size stops growing.
We also implemented the containertiming-ignore attribute. When a node has this attribute, it stops the SelfOrAncestorHasContainerTiming flag from propagating further down its subtree, so paints within it are not walked up to the container timing root, and never contribute to that container PaintedRegion.
Ignoring is useful for a number of things:
\nTip
containertiming-ignore on large untracked subtrees also reduces traversal depth, helping with the cost mentioned above.
With flag propagation, region aggregation, candidate buffering, and selective ignoring all in place, the implementation is complete.
\nContainer Timing is ready for test in Chromium. Just use the Blink feature flag ContainerTiming:
chrome --enable-blink-features=ContainerTimingBuilding this native implementation was a great exercise in reusing Blink’s existing performance infrastructure while extending it to support subtree-level aggregation.
\nThe key insight: subtree-level metrics didn’t require a new paint tracking system. Only a way to aggregate and bubble up what Blink was already measuring.
\nThe result is a native, low-overhead API for measuring the rendering performance of entire components.
\nThis has been done as part of the collaboration between Bloomberg and Igalia. Thanks!
\n\nOver the last year, as part of the collaboration between Igalia and Bloomberg to improve web performance observability, I worked on a new web performance API: Container Timing. This standard aims to make component-level performance measurement as easy as page-level metrics like LCP and FCP.
\nMy focus has been writing the native implementation in Chromium, which is now available behind a feature flag.
\nIn this post, I will explain why this API is needed, how it works, and how you can experiment with it today. In a follow-up post, I will dive deep into the implementation details within the Blink rendering engine.
\nWe currently use Largest Contentful Paint (LCP) and First Contentful Paint (FCP) to measure web page loading performance. Both metrics are page-scoped, meaning they evaluate the user perceived load speed for full page.
\nThe Element Timing API shifts the focus to individual DOM elements. By targetting specific elements, like hero images or a headers, we can measure their specific rendering performance independent of the rest of the page.
\nHowever, modern web development is component-based. Developers build complex widgets (as grids, charts, feeds or panels) that are made of many elements. It is not trivial to understand the performance of those components:
\n![\"A](\"https://blogs.igalia.com/dape/2026/02/10/container-timing-measuring-web-components-performance/images/container_timing_problem.png\")
This is where Container Timing comes in! With the new specification, a web developer can mark subtrees of the DOM as “containers”. Then, it provides performance entries aggregating the painting time of that subtree.
\n![\"A](\"https://blogs.igalia.com/dape/2026/02/10/container-timing-measuring-web-components-performance/images/container_timing_solution.png\")
This way, we can answer: “when did a specific component finish painting its content?”.
\nSome examples:
\nYou just need to add, to the top element of the subtree, the new attribute containertiming. When you add it to an HTML element, the browser will track all the painting updates of that element and its descendants.
What happens under the hood? The browser will start monitoring the rendering pipeline for paints that contribute to representing the subtree. When a new frame is painted, if that paints new areas for that subtree, it reports a performance entry showing the increase in painted area. It is similar to LCP, but for a specific subtree!
\nUsing the API is straightforward. First, mark the containers you want to track in HTML:
\n![\"Igalia\"](\"https://blogs.igalia.com/dape/img/igalia-logo-white-text.svg\"){kind=logo}
\n![\"Bloomberg\"](\"https://blogs.igalia.com/dape/img/Bloomberg-logo-580x117.png\"){kind=logo}
<div id=\"my-widget\" containertiming=\"widget-load\">
<img src=\"graph.png\" />
<p>Loading data...</p>
</div>Then, use a PerformanceObserver to listen for container entries:
const observer = new PerformanceObserver((list) => {
list.getEntries().forEach((entry) => {
console.log(`Container '${entry.identifier}' painted.`);
console.log(`Time: ${entry.startTime}`);
console.log(`Size: ${entry.size}`); // The area painted
});
});
observer.observe({ type: \"container\", buffered: true });When the web contents load, new Performance entries will be emitted with the container updates.
\nWhich entry will be interesting? The API lets you choose what best fits your needs! Some ideas:
\nIn the initial steps of the specification, Jason Williams wrote a polyfill that worked on top of Element Timing. This was very useful to understand and polish the kind of information the specification could provide. However, this had its own performance impact.
\nDeprecation Notice:
The polyfill is now deprecated and no longer maintained, as the native API cannot be fully replicated using Element Timing. Please use the native implementation for accurate results.
\nSo I started a native implementation in Chromium. The main idea was working on top of the already existing implementation for Element Timing, and add the remaining bits.
\nIn my next blog post I will go through the implementation details. But, for this post, it is relevant to state that the goals of this native implementation were:
\nThe native implementation has landed and is available in Chromium144+, but still behind the ContainerTiming feature flag.
You can experiment with this feature locally by passing the following flag to Chromium at startup:
\nchrome --enable-blink-features=ContainerTimingOr you can just enable the “Experimental Web Platform features” in chrome://flags.
So now, it is time to collect feedback from the actual web developers.
\nWe have already presented the specification in several conferences (as BlinkOn 20 or Performance.now() 2024). And discussions are ongoing in the Web Performance Working Group.
\nWe just announced the Dev Trial in the blink-dev mailing list! The feature is now officially ready for testing.
\nWhat’s next? We are also preparing an Origin Trial, that will allow developers to test the specification in production for a subset of their users.
\nIf you want to provide feedback, we are collecting it in the explainer ticket tracker.
\nWith Container Timing, you will be able to measure paintings at the web component level, filling a significant gap in the web performance monitoring landscape.
\nIf you struggled with finding out the ready time of your widgets, just try it! It is available, under the feature flags ContainerTiming, in Chromium Stable today.
And stay tuned! In a follow up post, I will go through the native implementation details in Chromium.
\nThis has been done as part of the collaboration between Bloomberg and Igalia. Thanks!
\n\nAs I write often, maintaining a downstream of Chromium is not easy. A lot of effort falls on the shoulders of the teams embedding Chromium, or creating products on top of the upstream Chromium project.
\nWe covered this in the previous chapters of my series of blog posts about maintaining Chromium downstreams. Now, this post is going to be a bit different.
\nI start with a question:
\n\n\nWhat can upstream Chromium do to help downstreams?
\n
This very same question was discussed in the Web Engines Hackfest breakout session that originated most of these posts. In this blog post, I will share some of the most interesting answers that came up in that session.
\nOne of the ideas was to move code around more aggressively to make it easier to reuse. Specifically, refactoring to move more and more code from //chrome to //components.
Chromium has gone a long way in that direction. Each of these changes allows downstreams to directly use only the components they need, instead of working on top of //chrome. But there is still room for improvement.
Some parts of //chrome are still not refactored and could be very useful, especially for downstreams shipping a browser. Some examples:
In the same direction, supporting easier ways to provide alternative implementations, and add custom software components, was considered important.
\nSome examples:
\n//chrome, would allow implementing new browsers without bundling the Chromium UI.A very interesting proposal was to create the role of downstream advocates in the Chrome community.
\nThey would act as an entry point for downstream projects wanting to interact with the Chrome community and be an official communication channel for downstreams to report their needs.
\nThis would also increase awareness of the different ways Chromium is used by downstreams.
\nToday there are two channels that are somewhat similar: the Chromium Embedders mailing list and the #embedders Slack channel.
So far, three different problems raised by downstreams have been covered, and they seem like fair requests to the Chromium community.
\nBut there is also work to do on the downstreams side.
\nCan downstreams contribute more of their work to upstream? Not only in code, but also in all the maintenance activities.
\nThere is also code written for very specific downstream needs that could land upstream, as long as it does not become a burden to the common project. That means ownership and enough work bandwidth need to be in place.
\nThere is a major change in the Chromium community: the creation of the Supporters of Chromium Based Browsers. What does it mean for embedders? Could it be a good way to channel requirements from the different downstream projects?
\nTwo years after the Web Engines Hackfest session, we can see some improvements. But the general question is still valid:
\n\n\nWhat can upstream Chromium do to help downstreams?
\n
The next post in this series will be the last one. It will cover some typical problems downstream projects are facing today.
\nAs part of my performance analysis work for LGE webOS, I often had to capture Chrome Traces from an embedded device. So, to make it convenient, I wrote a simple command line helper to obtain the traces remotely, named trace-chrome.
\nIn this blog post will explain why it is useful, and how to use it.
\nIf you want to read directly about the tool, jump to the how to use section.
\nChromium provides an infrastructure for capturing static tracing data, based on Perfetto. In this blog post I am not going through its architecture or implementation, but focus on how we instruct a trace capture to start and stop, and how to then fetch the results.
\nChrome/Chromium provides user interfaces for capturing and analyzing traces locally. This can be done opening a tab and pointing it to the chrome://tracing URL.
The tracing capture UI is quite powerful, and completely implemented in web. This has a downside, though: running the capture UI introduces a significant overhead in several resources (CPU, memory, GPU, …).
\nThis overhead may be even more significant when tracing Chromium or any other Chromium based web runtime in an embedded device, where we have CPU, storage and memory constraints.
\nChromium does a great work at minimizing the overhead, by postponing the trace processing as much as possible, and providing a minimal UI when the capture is ongoing. But it may still be too much.
\nHow to avoid this problem?
\nThe solution for both is tracing remotely. Both the user interface for controlling the recording, and the recording storage happen in a different computer.
\nFirst, some nomenclature I will use:
\nOK, now we know we want to trace remotely the target device Chromium instance. How can we do that? First, we need to connect our tracing tools running in the host to the Chromium instance in the target device.
\nThis is done using the remote debugging port: a multi-purpose HTTP port provided by Chromium. This port is used not only for tracing, it offers access to Chrome Developer Tools.
\nThe Chromium remote debugging port is disabled by default, but it can be enabled using the command line switch --remote-debugging-port=PORT in the target Chromium instance. This will open an HTTP port in the localhost interface, that can be used to connect.
Why localhost? Because this interface does not provide any authentication or encryption. So it is unsafe. It is user responsibility to provide some security (i.e. by using an setting an SSH tunnel between the host and the target device to connect to the remote debugging port).
Chromium browser provides a solution for tracing remotely. Just opening the URL chrome://inspect in the host device. It provides this user interface:
![\"\"](\"https://blogs.igalia.com/dape/2025/03/25/trace-chrome-easy-remote-tracing-of-chromium/images/chrome-inspect-1.png\")
First, the checkbox for Discover network targets needs to be set.
\nThen press the Configure… button to set the list of IP addressed and ports where we expect target remote debugging ports to be.
\nDo not forget to add to the list the end point that is accessible from the host device. I.e. in the case of an SSH tunnel from the host device to the target device port, it needs to be the host side of the tunnel.
\nFor the case we set up the host side tunnel at the port 10101, we will see this:
![\"\"](\"https://blogs.igalia.com/dape/2025/03/25/trace-chrome-easy-remote-tracing-of-chromium/images/chrome-inspect-2.png\")
Then, just pressing the trace link will show the Chromium tracing UI, but connected to the target device Chromium instance.
\ntrace-chrome #Over the last 8 years, I have been involved quite often in exploring the performance of Chromium in embedded devices. Specifically for the LGE webOS web stack. In this problem space, Chromium tracing capabilities are handy, providing a developers oriented view of different metrics, including the time spent running known operations in specific threads.
\nAt that time I did not know about chrome://inspect so I really did not have an easy way to collect Chromium traces from a different machine. This is important as one performance analysis principle is that collecting the information should be as lightweight as possible. Running the tracing UI in the same Chromium instance that is analyzed is against that principle.
The solution? I wrote a very simple NodeJS script, that allows to capture a Chromium trace from the command line.
\nThis is convenient for several reasons:
\nThe script just provides an easy to use command line interface to the already existing chrome-remote-interface NodeJS module.
The project is open source, and available at github.com/jdapena/trace-chrome.
\ntrace-chrome #Now, the instructions to use trace-chrome. The tool depends on having a working NodeJS environment in the host.
First, clone the Github repository in the host:
\ngit clone github.com:jdapena/trace-chromeThen, install the dependencies. To do this, you need to have a working NodeJS environment.
\ncd trace-chrome
npm installNow it is possible to try the tool. To get the command line help just run:
\n$ bin/trace-chrome --help
Usage: trace-chrome [options]
Options:
-H, --host <host> Remote debugging protocol host (default: \"localhost\")
-p, --port <port> Remote debugging protocool port (default: \"9876\")
-s, --showcategories Show categories
-O, --output <path> Output file (default: \"\")
-c, --categories <categories> Set categories (default: \"\")
-e, --excludecategories <categories> Exclude categories (default: \"\")
--systrace Enable systrace
--memory_dump_mode <mode> Memory dump mode (default: \"\")
--memory_dump_interval <interval_in_ms> Memory dump interval in ms (default: 2000)
--dump_memory_at_stop
-h, --help display help for commandTo connect to a running Chromium instance remote debugging port, the --host and --port parameters need to be used. In the examples I am going to use the port 9999 and the host localhost.
Warning
Note that, in this case, the parameter --host refers to the network address of the remote debugging port access point. It is not referring to the host machine where we run the script.
First, to check which tracing categories are available, we can use the option --showcategories:
bin/trace-chrome --host localhost --port 9999 --showcategoriesWe will obtain a list like this:
\nAccountFetcherService\nBlob\nCacheStorage\nCalculators\nCameraStream\n...\nNow, the most important step: recording a Chromium trace. To do this, we will provide a list of categories (parameter --categories), and a file path to record the trace (parameter --output):
bin/trace-chrome --host localhost --port 9999 \\
--categories \"blink,cc,gpu,renderer.scheduler,sequence_manager,v8,toplevel,viz\" \\
--output js_and_rendering.jsonThis will start recording. To stop recording, just press <Ctrl>+C, and the trace will be transferred and stored to the provided file path.
Tip
Which categories to use? Good presets for certain problem scopes can be obtained in Chrome. Just open chrome://tracing, press the Record button, and play with the predefined settings. In the bottom you will see the list of categories to pass for each of them.
Now the tracing file has been obtained, it can be opened from Chrome or Chromium running in host: load in a tab the URL chrome://tracing and press the Load button.
![\"\"](\"https://blogs.igalia.com/dape/2025/03/25/trace-chrome-easy-remote-tracing-of-chromium/images/chrome-tracing-load.png\")
Tip
The traces are completely standalone. So they can be loaded in any other computer without any additional artifact. This is useful, as those traces can be shared among developers or uploaded to a ticket tracker.
\nBut, if you want to do that, do not forget to compress first with gzip to make the trace smaller. chrome://tracing can open the compressed traces directly.
The script also supports periodical recording of the memory-infra system. This captures periodical dumps of the state of memory, with specific instrumentation in several categories.
To use it, add the category disabled-by-default-memory-infra, and pass the following parameters to configure the capture:
--memory_dump_mode <background|detailed|simple>: level of detail. background is designed to have almost no impact in execution, running very fast. light mode shows a few entries, while detailed is unlimited, and provides the most complete information.--memory_dump_interval: the interval in miliseconds between snapshots.For convenience, it is also possible to use trace-chrome with npx. It will install the script and the dependencies in the NPM cache, and run from them:
npx jdapena/trace-chrome --helpbin/trace-chrome --host HOST --port PORT \\
--categories \"blink,cc,netlog,renderer.scheduler,sequence_manager,toplevel,v8\" \\
--output web_developer.jsonbin/trace-chrome --host HOST --port PORT \\
--categories \"disabled-by-default-memory-infra\" --memory_dump_mode detailed \\
--memory_dump_interval 10000 --output memory_infra.jsontrace-chrome is a very simple tool, just providing a convenient command line interface for interacting with remote Chromium instances. It is specially useful for tracing embedded devices.
It has been useful for me for years, in a number of platforms, from Windows to Linux, from desktop to low end devices.
\nTry it!
\nMaintaining a downstream of Chromium is hard, because of the speed upstream moves. and how hard it is to keep our downstream up to date.
\nA critical aspect is how big what we build on top of Chromium is: in other words, the size of our downstream. In this blog post I will review how to measure it, and the impact it has on the costs of maintaining a downstream.
\nLast year, I started a series of blog posts about the challenges, the organization and the implementation details of maintaining a project that is a downstream of Chromium. This is the third blog post in the series.
\nThe previous posts were:
\nBut, first… What do I mean by the size of the downstream? I am interested in a definition that can be used as a metric, something we can measure and track. A number that allows to know if the downstream is increasing or decreasing, measure if a change has impact on it.
\nThe rough idea is: the bigger the downstream is, the more complex it is to maintain it. I will provide a few metrics that can be used for this purpose.
\nThe most obvious metric is the delta, the difference between upstream Chromium and the downstream. For this, and assuming the downstream uses Git, the definition I use is essentially the result of this command:
\ngit diff --shortstat BASELINE..DOWNSTREAMBASELINE is a commit reference that represents the pure upstream repository status our downstream is based on (our baseline). DOWNSTREAM is the commit reference we want to compare the baseline to.
As a recommendation, it is useful to maintain in our downstream repository tags or branches that represent strictly the baseline. This way we can use diff tools to represent our delta more easily.
\nThis command is going to return 3 values:
\nWe will be mostly interested in tracking the number of lines added and removed.
\nThis definition is interesting as it gives an idea of the amount of lines of code that we need to maintain. It may not reflect the full amount to maintain, as some files are maintained out of the Chromium repository. Aggregating these with other repositories changed or added to the build could be useful.
\nOne interesting thing with this approach is also that we can measure the delta of specific paths in the repository. I.e. if we want to measure the delta of the content/ path, it is just as easy as doing:
git diff --shortstat BASELINE..DOWNSTREAM content/The regular delta definition we considered has a problem. All the line changes have the same weight. But, when we update our baseline, a big part of the complexity comes from the conflicts found when rebasing or merging.
\nSo, I am introducing a new definition. Modifying delta: the changes between the baseline and the downstream that affect upstream lines. In this case, we ignore completely any file added only by the downstream, as that is not going to create conflicts.
\nIn Git, we can use filters for that purpose:
\ngit diff --diff-filter=CDMR --shortstat BASELINE..DOWNSTREAMThis will only account these changes:
\nM: changes affecting existing files.R: files that were renamed.C: files that were copied.D: files that were deleted.So, these numbers are going to more accurately represent which parts of our delta can conflict with the changes coming from upstream when we rebase or merge.
\nTracking the modifying delta, and reorganizing the project to reduce it, is usually a good strategy for reducing maintenance costs.
\nAn issue we have with the Git diff stats is that it represents modified lines as a block of lines removed and another of lines added.
\nFortunately, we can use another tool. Diffstat, will do a best effort to identify which lines are actually modified. It can be easily installed in your distribution of choice (i.e. the package diffstat in Debian/Ubuntu/Redhat).
This behavior is enabled with the parameter -m:
git diff ...parameters... | diffstat -mThis is the kind of output that is generated. On top of the typical + and - we see the ! for the lines that have been detected to be modified.
$ git show | diffstat -m
paint/timing/container_timing.cc | 5 ++++!
paint/timing/container_timing.h | 1 +
timing/performance_container_timing.cc | 20 ++++++++++++++++++!!
timing/performance_container_timing.h | 5 +++++
timing/performance_container_timing.idl | 1 +
timing/window_performance.cc | 4 ++!!
timing/window_performance.h | 1 +
7 files changed, 32 insertions(+), 5 modifications(!)Coloring is also available, with the parameter -C.
Using diffstat gives a more accurate insight of both the total delta and the modifying delta.
Now we have the tools to provide numbers, we can track them in the time to know if our downstream is growing or shrinking.
\nThat can be used also for measuring the impact of different strategies or changes in the downstream maintenance complexity.
\nBut deltas are not the only tool to measure the complexity, specially regarding the effort maintaining a downstream.
\nI can enumerate just a few ideas that provide insight of different problems:
\nLet’s focus now on other factors, not always measurable easily, when we maintain a downstream project.
\nThe complexity of a downstream, specially the one measured by regular delta, is impacted heavily by what is built on top of Chromium.
\nA full web browser is usually bigger, because it includes the required user experience, and many components that conform what we nowadays consider a browser. History, bookmarks, user profiles, secrets management…
\nAn application runtime for hybrid applications may just have minimal wrappers for integrating a web view, but then maybe a complex set of components for\neasing the integration with a native toolkit or a specific programming language.
\nHow much you build on top of Chrome?
\nFor maintenance complexity, as important as what we build on top, is the set of boundaries and dependencies:
\nThese questions are specially relevant, as Chromium does not really provide any warranty about the stability, or even availability, of existing components.
\nThough, different layers provided by Chromium change less often than others. Some examples:
\n//content/shell and //chrome.//components that may be useful for different downstreams.//chrome, and modify it for the specific downstream user experience. This means a higher modifying delta. But, as the upstream Chrome browser UI may also often changes heavily, the frequency of conflicts also increases.In this post I reviewed different ways to measure the downstream size, and how what we build impacts the complexity of maintenance.
\nUnderstanding and tracking our downstream allows to implement strategies to keep things under control. It also allows to better understand the cost of a specific feature or an implementation approach.
\nIn the next post in this series, I will write about how the upstream Chromium community helps the downstreams.
\nThis is the second of a series of blog posts I am publishing for sharing some considerations about the challenges of maintaining a downstream of Chromium.
\nThe first part, Maintaining downstreams of Chromium: why downstreams?, provided initial definitions, and an analysis of why someone would need to maintain a downstream.
\nIn this post I will focus on the possible different strategies for tracking the Chromium upstream project changes, and integrating the downstream changes.
\nThe first problem is related to the fact that our downstream will include additional changes on top of upstream (otherwise we would not need a downstream, right?).
\nThere are two different approaches for this, based on the Git strategy used: rebase vs. merge.
\nThis consists on maintaining an upstream branch and periodically merging its changes to the downstream branch.
\nIn the case of Chromium, the size of the repository and the amount and frequency of the changes is really big, so the chances that merging changes cause any conflict are higher than in other smaller projects.
\nThis consists on maintaining the downstream changes as a series of patches, that are applied on top of an upstream baseline.
\nUpdating means changing the baseline and applying all the patches. As this is done, not all patches will cause a conflict. And, for the ones that do, the complexity is far smaller.
\nDon’t hesitate to remove features that are more prone to cause conflicts. And at least, think carefully, for each change, if it can be done in a way that matches better upstream codebase.
\nAnd, if some of your changes are good for the commons, just contribute them to upstream!
\nA critical aspect of applying upstream changes is how often to do that.
\nThe size and structure of the team involved is highly dependent on this decision. And of course, planning the resources.
\nUsually this needs to be as much predictable as possible, and bound to the upstream release cycle.
\nSome examples used in downstream projects:
\nWeekly or daily tracking.
\nMajor releases.
\nIt is not only important how often you track upstream repository, but also what you track.
\nChromium, nowadays, follows this procedure for each major release (first number in the version):
\nMain development happens in main branch. Several releases are tagged daily. And, mostly every week, one of them is released to the dev channel for users to try (with additional testing).
At a point, an stabilization branch is created, entering beta stage. The quality bar for what is landed in this branch is raised. Only stabilization work is done. More releases per day are tagged, and again, approximately once per week one is released to beta channel.
When planned, the branch becomes stable. This means it is ready for wide user adoption, so the stable channel will pick its releases from this branch.
That means main (development branch) targets version N, then N-1 branch is the beta (or stabilization branch), and N-2 is the stable branch. Nowadays Chromium targets publishing a new major stable version every four weeks (and that also means a release spends 4 weeks in beta channel).
You can read the official release cycle documentation if you want to know more, including the extended stable (or long term support) channel.
Some downstreams will track upstream main branch. Some others will just start applying the downstream changes when first release lands the stable channel. And some may just start when main is branched and stabilization work begins.
The more you update, the smaller the change set you need to consider. And that means reducing the complexity of the changes.
\nThis is specially important when merging instead of rebasing, as applying the new changes is done in one step.
\nThis could be hard, though, depending on the size of the downstream. And, from time to time some refactors upstream will still imply non-trivial work to adapt the downstream changes.
\nOK, OK. I just told the opposite, right?
\nNow, think of a complex change upstream, that implies many intermediate steps, with a moving architecture that evolves as it is developed (something not unfrequent in Chromium). Updating often means you will need to adapt your downstream to all the intermediate steps.
\nThat definitely could mean more work.
\nIf you want to publish an stable release almost the same day or week of the official upstream stable release, and your downstream changes are not trivial, then a good moment for starting applying changes is when the stabilization branch happens.
But the first days there is a lot of stabilization work happening, as the quality bar is raised. So… waiting a few days after the beta branch happens could be a good idea.
An idea: just wait for the second version published in beta channel (the first one happens right after branching). That should give still three full weeks before the version is promoted to the stable channel.
In case you want to follow upstream main branch, in a daily basis (or even per commit), then it is just not practical to do that manually.
The solution for that is automation, at different levels:
\nApplying the changes.
\nResolving the conflicts.
\nAnd, more important, testing the result is working.
\nGood automation is, though, expensive. It requires computing power for building Chromium often, and run tests. But, increasing test coverage will detect issues earlier, and give an extra margin for resolving them.
\nIn any case, no matter your update strategy is, automation will always be helpful.
\nThis is the end of the post. So, what comes next?
\nIn the next post in this series, I will talk about the downstream size problem, and different approaches for keeping it under control.
\nChromium, the web browser open source project Google Chrome is based on, can be considered nowadays the reference implementation of the web platform. As such, it is the first choice when implementing the web platform in a software platform or product.
\nWhy is it like this? In this blog post I am going to introduce the topic, and then review the different reasons why a downstream Chromium is used in different projects.
\nThis is the first of a series of blog posts, where I am going through several aspects and challenges of maintaining a downstream project using Chromium as its upstream project.
\nThey will be mostly based on the discussions in two events. First, on The Web Engines Hackfest 2023 break out session with same title that I chaired in A Coruña. Then, on my BlinkOn 18 talk in November 2023, at Sunnyvale.
\nBefore starting the discussion of the different aspects, let’s clarify how I will use several terms.
\nI am going to refer to a repository as a version controlled storage of code strictly. Then, a project (specifically, a software project) is the community of people that share goals and some kind of organization, to maintain one or several software products.
\nSo, a project may use several repositories for their goals.
\nIn this discussion I will talk about Chromium, an open source project that targets the implementation of the web platform user agent, a web browser, for different platforms. As such, it uses a number of repositories (src, v8 and more).
I will use the downstream and upstream terms, referred to the relationship of different software projects version control repositories.
\nIf there is a software project repository (typically open source), and a new repository is created that contains all or part of the original repository, then:
\nIt is important to highlight that different things can happen to the downstream repository:
\nNowadays, web platform is a solid alternative for providing contents to the users. It allows modern user interfaces, based on well known standards, and integrate well with local and remote services. The gap between native applications and web contents has been reduced, so it is a good alternative quite often.
\nBut, when integrating web contents, product integrators need an implementation of the web platform. It is no surprise that Chromium is the most used, for a number of reasons:
\nStill, there are other good alternate choices for integrating the web, as WebKit (specially WPE for the embedded use cases), or using the system-provided web components (Android or iOS web view, …).
\nThough, in this blog post I will focus on the Chromium case.
\nBut, why do different projects need to use downstream Chromium repositories?
\nThe main simple reason the project needs a downstream repository is adding changes that are not upstream. This can be for a variety of reasons:
\nLet’s see some examples of changes of both types.
\nThis is when downstream adds support for a hardware target or OS that is not originally supported in upstream Chromium project.
\nChromium upstream provides an abstraction layer for that purpose named Ozone, that allows to adapt it to the OS, desktop environment, and system graphics compositor. But there are other abstraction layers for media acceleration, accessibility or input methods.
\nThe Wayland protocol adaptation started as a downstream effort, as upstream Chromium did not intend to support Wayland at that time. Eventually it evolved into an upstream official Ozone backend.
\nAn example? LGE webOS Chromium port.
\nThe previous case mostly forces to have a downstream project or repository. But there are also some cases where this is intended. There is the will to have some features in the downstream repository and not in upstream, an intended differentiation.
\nWhy would anybody want that? Some typical examples:
\nExamples of this:
\nAnd one last interesting case: integrating the web platform for developing hybrid applications: those that mix parts of the user interface implemented in a native toolkit, and parts implemented using the web platform.
\nThough Chromium includes official support for hybrid applications for Android, with the Android Web View, other toolkits provide also web applications support, and the integration of those belong, in Chromium case, to downstream projects.
\nExamples?
\nIn this blog post I presented different reasons why projects end up maintaining a downstream fork of Chromium.
\nIn the next blog post I will present one of the main challenges when maintaining a downstream of Chromium: the different rebase and upgrade strategies.
\n", "date_published": "2024-03-05T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2023/11/16/v8-profiling-instrumentation-overhead/", "url": "https://blogs.igalia.com/dape/2023/11/16/v8-profiling-instrumentation-overhead/", "title": "V8 profiling instrumentation overhead", "content_html": "In the last 2 years, I have been contributing to improve the performance of V8 profiling instrumentation in different scenarios, both for Linux (using Perf) and for Windows (using Windows Performance Toolkit).
\nMy work has been mostly improving the stack walk instrumentation that allows Javascript generated code to show in the stack traces, referring to the original code, in the same way native compiled code in C or C++ can provide that information.
\nIn this blog post I run different benchmarks to determine how much overhead this instrumentation introduces.
\nV8 implements system native profile instrumentation for the Javascript code, Both for Linux Perf and for Windows Performance Toolkit. Though, those are disabled by default.
\nFor Windows, the command line switch --enable-etw-stack-walking instruments the generated Javascript code, to emit the source code information for the compiled functions using ETW (Event Tracing for Windows). This gives a better insight of time spent in different functions, specially when stack profiling is enabled in Windows Performance Toolkit.
For Linux Perf, there are several command line switches: - --perf-basic-prof: this emits, for any Javascript generated code, its memory location and a descriptive name (that includes the source location of the function). - --perf-basic-prof-only-functions: same as the previous one, but only emitting functions, excluding other stubs as regular expressions. - --perf-prof: this is a different way to provide the information for Linux Perf. It generates a more complex format specified here. On top of the addresses of the functions, it also includes source code information, even with details about the exact line of code inside the function that is executed in a sample. - --perf-prof-annotate-wasm: this one extends --perf-prof, to add debugging information to WASM code. - --perf-prof-unwinding-info: This last one also extends --perf-prof, but providing experimental support for unwinding info.
And then, we have interpreted code support. V8 does not generate JIT code for everything, and in many cases it runs interpreted code that calls common builtin code. So, in a stack trace, instead of seeing the Javascript method that eventually runs those methods through the interpreter, we see those common methods. The solution? Using --interpreted-frames-native-stack, that adds additional information that identifies which Javascript method in the stack is actually interpreted. This is basic to understand the attribution of running code, especially if it is not considered hot enough to be compiled.
We would ideally want to have all of this profiling support always enabled when we are profiling code. For that, we would want to avoid having to pass any command line switch, so we can profile Javascript workloads without any additional configuration.
\nBut all these command line switches are disabled by default. Why? There are several reasons.
\nIn Linux, Perf profiling instrumentation will unconditionally generate additional files, as it cannot know if Perf is running or not. --perf-basic-prof backend writes the generated information to .map files, and --perf-prof backend writes additional information to jit-*.dump files that will be used later with perf inject -j.
Additionally, the instrumentation requires code compaction to be disabled. This is because code compaction will change the memory location of the compiled code. V8 generates CODE_MOVE internal events for profiler instrumentation. But ETW and Linux Perf backends do not handle those events for a variety of reasons. It has an impact in memory as there is no way to compact the space allocated for code without code move.
So, when we enable any of the Linux Perf command line switches we both generate extra files and take more memory.
\nIn Windows we have the same problem with code compaction. We do not support generating CODE_MOVE events so, while profiling, we cannot enable code compaction.
And the emitted ETW events with JIT code position information do still take memory space and make the profile recordings bigger.
\nBut there is an important difference: Windows ETW API allows applications to know when they are being profiled, and even filter that. V8 only emits the code location information for ETW if the application is being profiled, and code compaction is only disabled when profiling is ongoing.
\nThat means the overhead for enabling --enable-etw-stack-walking is expected to be minimal.
What about --interpreted-frames-native-stack? It has also been optimized to generate the extra stack information only when a profile is being recorded.
For Linux, the answer is a clear no. Any V8 workload would generate profiling information in files, and disabling code compaction makes memory usage higher. These problems happen no matter if you are recording a profile or not.
\nBut, Windows ETW support adds overhead only when profiling! It looks like it could make sense to enable both --interpreted-frames-native-stack and --enable-etw-stack-walking for Windows.
So first, we need to verify the actual overhead because we do not want to introduce a regression by enabling any of these options. I also want to confirm the actual overhead in Linux matches the expectation. To do that I am showing the result of running CSuite benchmarks, included as part of V8. I am capturing both the CPU and memory usage.
\nThe results obtained in Linux are…
\nLegend: - REC: recording a profile (Yes, No). - No flag: running the test without any extra command line flag. - BP: passing --perf-basic-prof. - IFNS: passing --interpreted-frames-native-stack. - BP+IFNS: passing both --perf-basic-prof and --interpreted-frames-native-stack.
Linux results - score:
\n| Test | \nREC | \nBetter | \nNo flag | \nBP | \nIFNS | \nBP+IFNS | \n
|---|---|---|---|---|---|---|
| Octane | \nNo | \nHigher | \n9101.2 | \n8953.3 | \n9077.1 | \n9097.3 | \n
| " | \nYes | \n" | \n9112.8 | \n9041.7 | \n9004.8 | \n9093.9 | \n
| Kraken | \nNo | \nLower | \n4060.3 | \n4108.9 | \n4076.6 | \n4119.9 | \n
| " | \nYes | \n" | \n4078.1 | \n4141.7 | \n4083.2 | \n4131.3 | \n
| Sunspider | \nNo | \nLower | \n595.7 | \n622.8 | \n595.4 | \n627.8 | \n
| " | \nYes | \n" | \n598.5 | \n626.6 | \n599.3 | \n633.3 | \n
Linux results - memory (in Kilobytes)
\n| Test | \nREC | \nNo flag | \nBP | \nIFNS | \nBP+IFNS | \n
|---|---|---|---|---|---|
| Octane | \nNo | \n244040.0 | \n249152.0 | \n243442.7 | \n253533.8 | \n
| " | \nYes | \n242234.7 | \n245010.7 | \n245632.9 | \n252108.0 | \n
| Kraken | \nNo | \n46039.1 | \n46169.5 | \n46009.1 | \n46497.1 | \n
| " | \nYes | \n46002.1 | \n46187.0 | \n46024.4 | \n46520.5 | \n
| Sunspider | \nNo | \n90267.0 | \n90857.2 | \n90214.6 | \n91100.4 | \n
| " | \nYes | \n90210.3 | \n90948.4 | \n90195.9 | \n91110.8 | \n
What is seen in the results? - There is apparently no CPU or memory overhead of --interpreted-frames-native-stack alone. Though there is an outlier while recording Octane memory usage, that could be an error in the sampling process. This is expected as the additional information is only generated while any profiling instrumentation is enabled. - There is no clear extra cost in memory for recording vs not recording. This is expected as the extra information only depends on having the switches enabled. It does not detect when recording is ongoing. - There is, as expected, a cost in CPU when Perf is ongoing, in most of the cases. - --perf-basic-prof has CPU and memory impact. And combined with --interpreted-frames-native-stack, the impact is even higher as expected.
This is on top of the fact that files are generated. The benchmarks back not enabling Perf --perf-basic-prof by default. But as --interpreted-frames-native-stack has only overhead in combination with profiling switches, it could make sense to consider enabling it on Linux.
What about Windows results?
\nLegend: - No flag: running the test without any extra command line flag. - ETW: passing --enable-etw-stack-walking. - IFNS: passing --interpreted-frames-native-stack. - ETW+IFNS: passing both --enable-etw-stack-walking and --interpreted-frames-native-stack.
Windows results - score:
\n| Test | \nREC | \nBetter | \nNo flag | \nETW | \nIFNS | \nETW+IFNS | \n
|---|---|---|---|---|---|---|
| Octane | \nNo | \nHigher | \n8323.8 | \n8336.2 | \n8308.7 | \n8310.4 | \n
| " | \nYes | \n" | \n8050.9 | \n7991.3 | \n8068.8 | \n7900.6 | \n
| Kraken | \nNo | \nLower | \n4273.0 | \n4294.4 | \n4303.7 | \n4279.7 | \n
| " | \nYes | \n" | \n4380.2 | \n4416.4 | \n4433.0 | \n4413.9 | \n
| Sunspider | \nNo | \nLower | \n671.1 | \n670.8 | \n672.0 | \n671.5 | \n
| " | \nYes | \n" | \n693.2 | \n703.9 | \n690.9 | \n716.2 | \n
Windows results - memory:
\n| Test | \nREC | \nNo flag | \nETW | \nIFNS | \nETW+IFNS | \n
|---|---|---|---|---|---|
| Octane | \nNot recording | \n202535.1 | \n204944.9 | \n200700.4 | \n203557.8 | \n
| " | \nRecording | \n205125.8 | \n204801.3 | \n206856.9 | \n209260.4 | \n
| Kraken | \nNot recording | \n76188.2 | \n76095.2 | \n76102.1 | \n76188.5 | \n
| " | \nRecording | \n76031.6 | \n76265.0 | \n76215.6 | \n76662.0 | \n
| Sunspider | \nNot recording | \n31784.9 | \n31888.4 | \n31806.9 | \n31882.7 | \n
| " | \nRecording | \n31848.9 | \n31882.1 | \n31802.7 | \n32339.0 | \n
What is observed? - No memory or CPU overhead is observed when recording is not ongoing, with --enable-etw-stack-walking and/or --interpreted-frames-native-stack. - Even recording, there is no clearly visible overhead of --interpreted-frames-native-stack. - When recording, --enable-etw-stack-walking has an impact on CPU. But not on memory. - When --enable-etw-stack-walking and --interpreted-frames-native-stack are combined, while recording, both memory and CPU overheads are observed.
So, again, it should be possible to enable --interpreted-frames-native-stack on Windows by default as it only has impact while recording a profile with --enable-etw-stack-walking. And it should be possible to enable --enable-etw-stack-walking by default too as, again, the impact happens only when a profile is recorded.
Is it that simple? Is it just OK only adding overhead when recording a profile?
\nOne problem with this is that there is an impact in system wide recordings, even if the focus is not on Javascript execution.
\nAlso, the CPU impact is not linear. Most of it happens when a profile recording starts, V8 emits the code position of all the methods in all the already existing Javascript functions and builtins. Now, imagine a regular desktop with several V8 based browsers as Chrome and Edge, with all their tabs. Or a server with many NodeJS instances. All that generates the information of all their methods at the same time.
\nSo the impact of the overhead is significant, even if it only happens while recording.
\nFor --interpreted-frames-native-stack it looks like it would be better to propose enabling it by default in all studied platforms (Windows and Linux). It significantly improves the profiling instrumentation, and it has impact only when actual instrumentation is used. And then it still allows disabling it for the specific cases where the original builtins recording is preferred.
For --enable-etw-stack-walking, it could make sense to also propose enabling it by default, even with the known overhead while recording profiles. And just make sure there are no surprise regressions.
But, likely, the best option is trying to reduce that overhead. First, by allowing to filter better which processes generate the additional information from Windows Performance Recorder. And then, also, by reducing the initial overhead as much as possible. I.e. a big part of it is emitting the builtins and V8 snapshot compiled functions for each of the contexts again and again. Ideally we want to avoid that duplication if possible.
\nWhile on Linux, enabling profile instrumentation with command line switches is not an option, Windows dynamically enabling instrumentation allows to consider enabling ETW support by default. But there are still optimization opportunities that could be addressed before.
\nThis report has been done as part of the collaboration between Bloomberg and Igalia. Thanks!
\n\n", "date_published": "2023-11-16T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2023/11/06/keep-gcc-running-2023-update/", "url": "https://blogs.igalia.com/dape/2023/11/06/keep-gcc-running-2023-update/", "title": "Keep GCC running (2023 update)", "content_html": "Last week I attended BlinkOn 18 in Sunnyvale Google offices. For the 5th time I presented the status of Chromium build using the GNU toolchain components: GCC compiler and libstdc++ standard library implementation.
\nThis blog post recaps the current status in 2023, as it was presented in the conference.
\nFirst things first: GCC is still maintained, and working, on the official development releases! Though, as official build bots will not check that, fixes usually take a few extra days to land.
\nThis is the result of the work from contributors of several companies (Igalia, LGE, Vewd and others). But most important, from individual contributors (last 2 years the main contributor, with more than 50% of the commit has been Stephan Hartmann from Gentoo).
\nThe work to support GCC is coordinated from the GCC Chromium meta bug.
\nSince I started tracking the GCC support we have been getting a quite stable number of contributions, 70-100 per year. Though in 2023 (even if we are counting only 8 months on this chart), we are way below 40.
\n![\"\"](\"https://blogs.igalia.com/dape/2023/11/06/keep-gcc-running-2023-update/images/gcc_stats_2023-1.png\")
What happened? I am not really sure. Though, I have some ideas. First, simply as we move to newer versions of GCC, the implementation of recent C++ standards have improved. But then, also, this is the result of the great work that has been done recently in both GCC and LLVM, and also in standarization process, to get more interoperable implementations.
\nNow, I will focus on the most frequent causes for build breakage affecting GCC since January 2022.
\nThe rules for visibility in C++ are slightly different in GCC. An example: if a class declares a getter with the same name of a declared type in current namespace, Clang will be mostly Ok with that. But GCC will fail with an error.
\nBad:
\nusing Foo = int;
class A {
...
Foo Foo();
...
}A possible fix is renaming the accessor method name:
\nFoo GetFoo();Or using a explicit namespace for the type:
\n::Foo GetFoo()GCC may sometimes fail to resolve which constructor to use when there is an implicit type conversion. To avoid that, we can make that conversion explicit or use all braces initializers.
\nconstexpr is more strict in GCC #In GCC a constexpr method declared as default demands all its parts to be also declared constexpr:
Bad
\nint MethodA() { ... };
constexpr MethodB() { ... MethodA() ... };Two possible fixes: or dropping constexpr from the using method, or adding constexpr to all used methods.
noexcept strictness #noexcept strictness in GCC and Clang is the same when exceptions are enabled, requiring that all invoked functions used from a noexcept are also noexcept. Though, when exceptions are disabled using -fno-exception, Clang will ignore the errors. But GCC will still check the rules.
This works in Clang and not in GCC if -fno-exception is set:
class A {
...
virtual int Method();
};
class B {
...
int Method() noexcept override;
};Implicit casts of CPU intrinsic types will fail in GCC, requiring explicit conversion or cast.
\nAs example:
\nint8_t input __attribute__((vector_size(16)));
uint16_t output = _mm_movemask_epi8(input);If we see the declaration of the intrinsic call:
\nint _mm_movemask_epi8 (__m128i a);GCC will not allow implicit casting from an int8_t __attribute__((vector_size(16))), though the storage is the same. In GCC we require a reinterpret_cast:
int8_t input __attribute__((vector_size(16)));
uint16_t output = _mm_movemask_epi8(reinterpret_cast<__m128i>(input));Template specializations are not allowed in GCC if they are not in a namespace scope. Usually this error materializes with developers adding the template specialization in the templated class scope.
A failing example:
\nnamespace a {
class A() {
template<typename T>
T Foo(const T&t ) {
...
}
template<>
size_ Foo(const size_t& t)
};
}The fix is moving the template specialization to the namespace scope:
\nnamespace a {
class A() {
template<typename T>
T Foo(const T&t ) {
...
}
};
template<>
size_ A::Foo(const size_t& t) {
...
}
}Regarding the C++ standard library implementation from GNU project, the work is coordinated in the libstdc++ Chromium meta bug.
\nWe have definitely observed a big increase of required fixes in 2022, and projection of 2023 is going to be similar.
\n![\"\"](\"https://blogs.igalia.com/dape/2023/11/06/keep-gcc-running-2023-update/images/libstdcxx_stats_2023-1.png\")
In this case there are two possible reasons. One is that we are more exhaustively tracking the work using the meta bug.
\nIn the case of libstdc++, these are the most frequent causes for build breakage since January 2022.
\nThe libc++ implementation is different from libstdc++. And that implies some library headers that could be indirectly included in libc++ are not in libstdc++.
\nLet’s see this code:
\nstd::vector<const int> v;In Clang, this is allowed. Though, in GCC there is an explicit assert forbidding it: std::vector must have a non-const, non-volatile value_type.
This is not only specific to std::vector, but also to std::unordered:*, std::list, std::set, std::deque, std::forward_list or std::multiset.
The solution? Just do not use const as members:
\nstd::vector<int> v;Assigning from nullptr to std::unique_ptr requires destructor declaration
\nclass A;
std::unique_ptr<A> GetValue() {
return nullptr;
}Usually it is just needed to include the full declaration of the class, as it needs the size of the type for the default destructor.
\nIn my case, to verify GCC and libstdc++ support, on top of building Chromium in Ubuntu using both, I am also maintaining a Yocto layer that builds Chromium development releases. I usually try to verify the build in less than a week after each release.
\nThe layer is available at github.com/Igalia/meta-chromium
\nI am regularly testing the build using core-image-weston in both Raspberry PI 4 (64 bits) and Intel x86-64 (using Qemu). There I try both X11 and Wayland Ozone backends.
I would still recommend people to move to use the officially supported Chromium toolchains: libc++ and Clang. With them, downstreams get a far more tested implementation, more security features, or better integration for sanitizers. But, meanwhile, I expect things will still be kept working as several downstreams and distributions will still ship Chromium on top of the GNU toolchains.
\nEven with the GNU toolchain not being officially supported in Chromium, the community has been successful providing support for both GCC and libstdc++. Thanks to all the contributors!
\n", "date_published": "2023-11-06T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2023/07/28/speeding-up-v8-heap-snapshot/", "url": "https://blogs.igalia.com/dape/2023/07/28/speeding-up-v8-heap-snapshot/", "title": "Speeding up V8 heap snapshot", "content_html": "My last post, Javascript memory profiling with heap snapshot, finished announcing I would write a follow up post about several optimizations I implemented that make heap snapshot faster.
\nGood news! The post has been accepted in V8.dev! You can read it here.
\n", "date_published": "2023-07-28T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2023/05/18/javascript-memory-profiling-with-heap-snapshot/", "url": "https://blogs.igalia.com/dape/2023/05/18/javascript-memory-profiling-with-heap-snapshot/", "title": "Javascript memory profiling with heap snapshot", "content_html": "In both web and NodeJS worlds, the main runtime for executing program logic is the Javascript runtime. Because of that, a huge number of applications and user interfaces are using it. As any software component, Javascript code uses resources of the system, that are not unlimited. We should be careful when using CPU time, application storage, or memory.
\nIn this blog post we are going to focus on the latter.
\nUsually the objects allocated by a web page are not a lot, so they do not eat a huge amount of memory for a modern and beefy computer. But we find problems like:
\nThe user may not be happy with the web application developer implementation choices. And this developer may want to be… more efficient. Do something.
\nNow… Think about the cloud providers. And developers implementing software using NodeJS in the cloud. The contract with the provider may limit the available memory… Or get money depending on the actual usage.
\nSo… An innocent script that takes 10MB, but is run thousands or millions or times for a few seconds. That is expensive!
\nThese developers will need to make their apps… again, more efficient.
\nIn performance problems, we usually want to have reliable data of what is happening, and when. Memory problems are no different. We need some observability of the memory usage.
\nChromium and NodeJS share their Javascript runtime, V8, and it provides some tools to help with memory investigation.
\nIn this post we are going to focus on the family of tools around a V8 feature named heap snapshot, that allows capturing the memory usage at any time in a Javascript execution context.
\n:!: This is a fast recap on how Javascript heap works, you can skip it if you want
\nIn V8 Javascript runtime, variables, no matter their scope, are allocated on a heap. No matter if it is a number, a string, an object or a function, all of them are stored there. Not only that, in V8 even the code is stored in the heap.
\nBut, in Javascript, memory is freed lazily, with a garbage collection. This means that, when an object is not used anymore, its memory is not immediately disposed. Garbage collector will explore which objects are disposable later, and free them when it is convenient.
\nHow do we know if an object is still used? The idea is simple: objects are used if they can be accessed. To find out which ones, the runtime will take the root objects, and explore recursively all the object references. Any object that has not been found in that exploration can be discarded.
\nOK, and what is a root object? In a script it can be the objects in the global context. But also Javascript objects referred from native objects.
\nMore details of how the V8 garbage collector works are out of the scope of this post. If you want to learn more, this post should provide a good overview of current implementation: Trash talk: the Orinoco garbage collector.
\nOK, so we know all the Javascript memory allocation goes through the heap. And, as I said, heap snapshot is a tool to investigate memory problems.
\nThe name is quite explicit about how it works. Heap snapshot will stop the Javascript execution, traverse all the heap, analyze it, and dump it in a meaningful format that can be investigated.
\nWhat kind of information does it have?
\nThe format of those snapshots is using JSON, and it can be opened from Chromium developer tools for analysis.
\nIn the Chromium browser, heap snapshots can be obtained from the Chrome developer tools, accessed through the Inspect right button menu option.
\nThis is common to any browser based in Chromium exposing those developer tools locally or remotely.
\nOnce the developer tools are visible, there is the Memory tab:
\n![\"\"](\"https://blogs.igalia.com/dape/2023/05/18/javascript-memory-profiling-with-heap-snapshot/images/Captura-de-pantalla-de-2023-05-12-14-35-17.png\")
We can select three profiling types:
\nIn all cases, we will get a profiling report that we can analyze later.
\n![\"\"](\"https://blogs.igalia.com/dape/2023/05/18/javascript-memory-profiling-with-heap-snapshot/images/Captura-de-pantalla-de-2023-05-12-14-36-12.png\")
In NodeJS, we can attach the Chrome dev tools passing --inspect through the command line or the NODE_OPTIONS environment variable. This will attach the inspector to NodeJS, but it does not stop execution. The variant --inspect-brk will break on debugger at start of the user script.
How does it work? It will open a port in localhost:9229, and then this can be accessed from Chromium browser URL chrome://inspect. The UI allows users to select which hosts to listen to for Node sessions. The end point can be modified using --inspect=[HOST:]PORT, --inspect-brk=[HOST:]PORT or with the specific command line argument --inspect-port=[HOST:]PORT.
Once you attach dev tools inspector, you can access the Memory tab as in the case of Chromium
\nThere is a problem, though, when we are using NODE_OPTIONS. All instances of NodeJS will take the same parameter, so they will try to attach to the same host and port. And only the first instance will get the port. So it is less useful than we would expect for a session running multiple NodeJS processes (as it can be just running NPM or YARN to run stuff).
Oh, but there are some tricks!:
\n0 it will allocate a port (and report it through the console!). So you can inspect any arbitrary session (more details).SIGUSR1. This will run in default localhost:9229 unless a different setting is specified with --inspect-port=[HOST:]PORT (more details).Also, there are other ways to obtain heap snapshots directly, without using developer tools UI. NodeJS allows to pass different command line parameters for programming heap snapshot capture/profiling:
\n--heapsnapshot-near-heap-limit=N will dump a heap snapshot when the V8 heap is close to its maximum size limit. The N parameter is the number of times it will dump a new snapshot. This is important because, when V8 is reaching the heap limit, it will take measures to free memory through garbage collection, so in a pattern of growing usage we will hit the limit several times.--heapsnapshot-signal=SIGNAL will dump heap snapshots every time the NodeJS process gets the UNIX signal SIGNAL.We can also record a heap profiling session from the start of the process to the end (same kind of profiling we obtain from Dev Tools using Allocation sampling option) using command line option --heap-prof. This will sample continuously the memory allocations, and can be tuned using different command line parameters as documented here.
The scope of this post is about how to capture heap snapshots in different scenarios. But… once you have them… You will want to use that information to actually understand memory usage. Here are some good reads about how to use heap snapshots.
\nFirst, from Chrome DevTools documentation:
\nAnd then, from NodeJS, you have also a couple of interesting things:
\nIn a follow up post, I will talk about several optimizations we worked on, that make V8 heap snapshot implementation faster. Stay tuned!
\nThis work has been thanks to the sponsorship from Igalia and Bloomberg.
\n\n", "date_published": "2023-05-18T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2023/03/14/stack-walk-profiling-nodejs-in-windows/", "url": "https://blogs.igalia.com/dape/2023/03/14/stack-walk-profiling-nodejs-in-windows/", "title": "Stack walk profiling NodeJS in Windows", "content_html": "Last year I wrote a series of blog posts (1, 2, 3) about stack walk profiling Chromium using Windows native tools around ETW.
\nA fast recap: ETW support for stack walking in V8 allows to show V8 JIT generated code in the Windows Performance Analyzer. This is a powerful tool to analyze work loads where Javascript execution time is significant.
\nIn this blog post, I will cover the usage of this very same tool, but to analyze NodeJS execution.
\nIn an ideal situation, V8 engines would always generate stack walk information when Windows is profiling. This is something we will want to consider in the future, as we prove enabling it has no cost if we are not in a tracing session.
\nMeanwhile, we need to set the V8 flag --enable-etw-stack-walking somehow. This will install hooks that, when a profiling session starts, will emit the JIT generated code addresses, and the information about the source code associated to them.
For a command line execution of NodeJS runtime, it is as simple as passing the command line flag:
\nnode --enable-etw-stack-walkingThis will work enabling ETW stack walking for that specific NodeJS session… Good, but not very useful.
\nWhat’s the problem here? Usually, NodeJS is invoked indirectly through other tools (based or not in NodeJS). Some examples are Yarn, NPM, or even some Windows scripts or link files.
\nWe could tune all the existing launching scripts to pass --enable-etw-stack-walking to the NodeJS runtime when it is called. But that is not much convenient.
There is a better way though, just using NODE_OPTIONS environment variable. This way, stack walking support can be enabled for all NodeJS calls in a shell session, or even system wide.
Some bad news: NodeJS was refusing --enable-etw-stack-walking in NODE_OPTIONS. There is a filter for which V8 options it accepts (mostly for security purposes), and ETW support was not considered.
Good news? I implemented a fix adding the flag to the list accepted by NODE_OPTIONS. It has been landed already, and it is available from NodeJS 19.6.0. Unfortunately, if you are using an older version, then you may need to backport the patch.
\nTo explain how this can be used, I will analyse ESLint on a known workload: TypeScript. For simplicity, we are using the lint task provided by TypeScript.
This example assumes the usage of Git Bash.
\nFirst, clone TypeScript from GitHub, and go to the cloned copy:
\ngit clone https://github.com/microsoft/TypeScript.git
cd TypeScriptThen, install hereby and the dependencies of TypeScript:
\nnpm install -g hereby
npm ciNow, we are ready to profile the lint task. First, set NODE_OPTIONS:
export NODE_OPTIONS=\"--enable-etw-stack-walking\"Then, launch UIForETW. This tool simplifies capturing traces, and will provide good defaults for Javascript ETW analysis. It provides a very useful keyboard shortcut,
Switch to Git Bash terminal and do this sequence:
\nhereby lintAfter a few seconds UIForETW will automatically open the trace in Windows Performance Analyzer. Thanks to settings NODE_OPTIONS all the child processes of the parent node.exe execution also have stack walk information.
Focusing on node.exe instances, in Randomascii inclusive (stack) view, we can see where time is spent for each of the node.exe processes. If I take the bigger one (that is the longest of the benchmarks I executed), I get some nice insights.
The worker threads take 40% of the CPU processing. What is happening there? I basically see JIT compilation and garbage collection concurrent marking. V8 offloads that work, so there is a benefit from a multicore machine.
\n![\"\"](\"https://blogs.igalia.com/dape/2023/03/14/stack-walk-profiling-nodejs-in-windows/images/worker-thread-compile-and-gc.png\")
Most of the work happens in the main thread, as expected. And most of the time is spent parsing and applying the lint rules (half for each).
\n![\"\"](\"https://blogs.igalia.com/dape/2023/03/14/stack-walk-profiling-nodejs-in-windows/images/main-thread-rules-and-parse.png\")
If we go deeper in the rules processing, we can see which rules are more expensive.
\n![\"\"](\"https://blogs.igalia.com/dape/2023/03/14/stack-walk-profiling-nodejs-in-windows/images/lint-rules-error-handler.png\")
In total commit view, we can observe the memory usage pattern of the process running ESLint. For most of the seconds of the workload, allocation grows steadily (to over 2GB of RAM). Then there is a first garbage collection, and a bit later, the process finishes and all the memory is deallocated.
\n![\"\"](\"https://blogs.igalia.com/dape/2023/03/14/stack-walk-profiling-nodejs-in-windows/images/memory-committed.png\")
At first sight, I observe we are creating the rules objects for all the execution of ESLint. What does it mean? Could we run faster reusing those? I can also observe that a big part of the time in main thread leads to leaves doing garbage collection.
\nThis is a good start! You can see how ETW can give you insights of what is happening and how much time it takes. And even correlate that to memory usage, File I/O, etc.
\nUsing NodeJS, as is today, will still show many missing lines in the stack. I did those tests, and could do a useful analysis, because I applied a very recent patch I landed in V8.
\nBefore the fix, we would have this sequence:
\nBut there was a problem: any JS context has already a lot of pre-compiled code associated: builtins and V8 snapshot code. Those were missing from the ETW traces captured.
\nThe fix, as said, has been already landed to V8, and hopefully will be available soon in future NodeJS releases.
\nThere is more work to do:
\n--enable-etw-stack-walking set by default, as the impact while not tracing is minimal.In any case, after these new fixes, capturing ETW stack walks of code executed by NodeJS runtime is a bit easier. I hope this gives some joy to your performance research.
\nOne last thing! My work for these fixes is possible thanks to the sponsorship from Igalia and Bloomberg.
\n", "date_published": "2023-03-14T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2022/12/21/native-call-stack-profiling-3-3-2022-work-in-v8/", "url": "https://blogs.igalia.com/dape/2022/12/21/native-call-stack-profiling-3-3-2022-work-in-v8/", "title": "Native call stack profiling (3/3): 2022 work in V8", "content_html": "This is the last blog post of the series. In first post I presented some concepts of call stack profiling, and why it is useful. In second post I reviewed Event Tracing for Windows, the native tool for the purpose, and how it can be used to trace Chromium.
\nThis last post will review the work done in 2022 to improve the support in V8 of call stack profiling in Windows.
\nI worked on several of the fixes this year. This work has been sponsored by Bloomberg and Igalia.
\nThis work was presented as a lightning talk in BlinkOn 17.
\nIn March I started working on the report that Windows event traces where not properly resolving the Javascript symbols.
\nAfter some bisecting I found this was a regression introduced by this commit, that changed the --js-flags handling to a later stage. This happened to be after V8 initialization, so the code that would enable instrumentation would not consider the flag.
The fix I implemented moved flags processing to happen right before platform initialization, so instrumentation worked again.
\nAnother fix I worked was to improve the methods name generation. Windows tracing would show a quite redundant description of each level, and that was making analysis more difficult.
\nBefore my work, the entries would look like this:
\nstring-tagcloud.js!LazyCompile:~makeTagCloud- string-tagcloud.js:231-232:22 0x0\nAfter my change, now it looks like this:
\nstring-tagcloud.js!makeTagCloud-231:22 0x0\nThe fix adds a specific implementation for ETW. Instead of reusing the method name that is also used for Perf, it has a specific implementation for function that takes into account what ETW backend exports already, to avoid redundancy. It also takes advantage of the existing method DebugNameCStr to retrieve inferred method names in case there is no name available.
The way V8 ETW worked was that, when tracing was ongoing and a new function was compiled in JIT, it would emit information to ETW.
\nThis implied a big problem. If a function was compiled by V8 before tracing started, then ETW would not properly resolve the function names so, when analyzing the traces, it would not be possible to know which function was called at any of the samples.
\nThe solution is conceptually simple. When tracing starts, V8 traverse the living Javascript contexts and emit all the symbols. This adds noise to the tracing, as it is an expensive process. But, as it happens at the start of the tracing, it is very easy to isolate in the captured trace.
\nI also fixed a huge performance penalty when tracing code from snapshots, caused by calculating all the time the end line numbers of code instead of caching it.
\nPaolo Severini improved the initialization code, so the initialization of an ETW session was lighter, and also tracing would be started or stopped correctly.
\nAfter all these changes I did some benchmarking with and without ETW. The goal was knowing if it would be good to enable by default ETW support in V8, not requiring to pass any JS flag.
\nWith Sunspider in a Windows 64 bits build:
\n![\"Image](\"https://blogs.igalia.com/dape/2022/12/21/native-call-stack-profiling-3-3-2022-work-in-v8/images/etw-and-interpreted-frames-native-stack-overhead.svg\")
Other benchmarks I tried gave similar numbers.
\nSo far, in 64 bits architecture I could not detect any overhead of enabling ETW support when recording is not happening, and the cost when it is enabled is very low.
\nThough, when combined with interpreted frames native stack, the overhead is close to 10%. This was expected as explained here.
\nSo, good news so far. We still need to benchmark 32 bit architecture to see if the impact is similar.
\nThe work described in this post is available in V8 10.9.0. I hope you enjoy the improvements, and specially hope these tools help in the investigation of performance issues around Javascript, in NodeJS, Google Chrome or Microsoft Edge.
\nThere is still a lot of things to do, and I hope I can continue working on improvements for V8 ETW support next year:
\nIf you want to track the work, keep an eye on V8 issue 11043.
\nThis is the last post in the series.
\nThanks to Bloomberg and Igalia for sponsoring my work in ETW Chromium integration improvements!
\n", "date_published": "2022-12-21T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2022/11/29/native-call-stack-profiling-2-3-event-tracing-for-windows-and-chromium/", "url": "https://blogs.igalia.com/dape/2022/11/29/native-call-stack-profiling-2-3-event-tracing-for-windows-and-chromium/", "title": "Native call stack profiling (2/3): Event Tracing for Windows and Chromium", "content_html": "In last blog post, I introduced call stack profiling, why it is useful, and how a system wide support can be useful. This new blog post will talk about Windows native call stack tracing, and how it is integrated in Chromium.
\nEvent Tracing for Windows, usually also named with the acronym ETW, is a Windows kernel based tool that allows to log kernel and application events to a file.
\nA good description of its architecture is available at Microsoft Learn: About Event Tracing.
\nEssentially, it is an efficient event capturing tool, in some ways similar to LTTng. Its events recording stage is as lightweight as possible to avoid processing of collected data impacting the results as much as possible, reducing the observer effect.
\nThe main participants are:
\nWhat about call stack profiling? ETW supports call stack sampling, allowing to capture call stacks when certain events happen, and associates the call stack to that event. Bruce Dawson has written a fantastic blog post about the topic.
\nChromium provides support for call stack profiling. This is done at different levels of the stack:
\nIn any platform compilation will usually benefit from compiling with the GN flag enable_profiling=true. This will enable frame pointers support. In Windows, it will also enable generation of function information for code generated by V8.
Also, symbol_level=1 should be added at least, so the compilation stage function names are available.
To enable generation of V8 profiling information in Windows, these flags should be passed to chrome on launch:
\nchrome --js-flags=\"--enable-etw-stack-walking --interpreted-frames-native-stack\"--enable-etw-stack-walking will emit information of the functions compiled by V8 JIT engine, so they can be recorded while sampling the stack.
--interpreted-frames-native-stack will show the frames of interpreted code in the native stack, so external profilers as ETW can properly show those in the profiling samples.
Then, a session of the workload to analyze can be captured with Windows Performance Recorder.
\nAn alternate tool with specific Chromium support, UIForETW can be used too. Main advantage is that it allows to select specific Chromium tracer categories, that will be emitted in the same trace. Its author, Bruce Dawson, has explained very well how to use it.
\nFor analysis, the tool Windows Performance Analyzer (WPA) can be used. Both UIForETW and Windows Performance Recorder will offer opening the obtained trace at the end of the capture for analysis.
\nBefore starting analysis, in WPA, add the paths where the .PDB files with debugging information are available.
\nThen, select Computation/CPU Usage (Sampled):
\n![\"\"](\"https://blogs.igalia.com/dape/2022/11/29/native-call-stack-profiling-2-3-event-tracing-for-windows-and-chromium/images/wpa_cpu_usage_sampled.png\")
.
From the available charts, we are interested in the ones providing stackwalk information:
\n![\"\"](\"https://blogs.igalia.com/dape/2022/11/29/native-call-stack-profiling-2-3-event-tracing-for-windows-and-chromium/images/wpa_cpu_usage_sampled_stack_examples.png\")
In the last post of this series, I will present the work done in 2022 to improve V8 support for Windows ETW.
\n", "date_published": "2022-11-29T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2022/11/16/native-call-stack-profiling-1-3-introduction/", "url": "https://blogs.igalia.com/dape/2022/11/16/native-call-stack-profiling-1-3-introduction/", "title": "Native call stack profiling (1/3): introduction", "content_html": "This week I presented a lightning talk in BlinkOn 17. There I talked about the work for improving native stack profiling support in Windows.
\nThis post starts a series where I will provide more context and details to the presentation.
\nFirst, a definition:
\nCallstack profiling: a performance analysis tool, that samples periodically the call stacks of all threads, for a specific workload.
\nWhy is it useful? It provides a better undestanding of performance problems, specially if they are caused by CPU-bound bottle necks.
\nAs we sample the full stack for each thread, we are capturing a handful of information:
\nBut it is not only useful for CPU waits. It will also capture when a method is waiting for something (i.e. because of networking, or a semaphore).
\nThe provided information is useful for initial analysis of the problem, as it will give a high level view of where time could be spent by the application. But it will also be useful in further stages of the analysis, and even for comparing different implementations and consider possible changes.
\nFor call stack sampling, we need some infrastructure to be able to capture and traverse properly the callstack for each thread.
\nIn compilation stage, information is added for function names and the frame pointers. This allows, for a specific stack, to resolve later the actual names, and even lines of code that are captured.
\nIn runtime stage, function information will be required for generated code. I.e. in a web browser, the Javascript code that is compiled in runtime.
\nThen, every sample will extract the callstack of all the threads of all the analysed processes. This will happen periodically, at the rate established by the profiling tool.
\nWhen possible, sampling the call stacks of the full system can be benefitial for the analysis.
\nFirst, we may want to include system libraries and other dependencies of our component in the analysis. But also, system analyzers can provide other metrics that can give a better context to the analysed workload (network or CPU load, memory usage, swappiness, …).
\nIn the end, many problems are not bound to a single component, so capturing the interaction with other components can be useful.
\nIn next blog posts in this series, I will present native stack profiling for Windows, and how it is integrated with Chromium.
\n", "date_published": "2022-11-16T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/", "url": "https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/", "title": "3 events in a month", "content_html": "As part of my job at Igalia, I have been attending 2-3 events per year. My role mostly as a Chromium stack engineer is not usually much demanding regarding conference trips, but they are quite important as an opportunity to meet collaborators and project mates.
\nThis month has been a bit different, as I ended up visiting Santa Clara LG Silicon Valley Lab in California, Igalia headquarters in A Coruña, and Dresden. It was mostly because I got involved in the discussions for the web runtime implementation being developed by Igalia for AGL.
\nIt is always great to visit LG Silicon Valley Lab (Santa Clara, US), where my team is located. I have been participating for 6 years in the development of the webOS web stack you can most prominently enjoy in LG webOS smart TV.
\nOne of the goals for next months at AGL is providing an efficient web runtime. In LGSVL we have been developing and maintaining WAM, the webOS web runtime. And as it was released with an open source license in webOS Open Source Edition, it looked like a great match for AGL. So my team did a proof of concept in May and it was succesful. At the same time Igalia has been working on porting Chromium browser to AGL. So, after some discussions AGL approved sponsoring my company, Igalia for porting the LG webOS web runtime to AGL.
\nAs LGSVL was hosting the september 2018 AGL f2f meeting, Igalia sponsored my trip to the event.
\n![](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/img_4005.jpg\")
\nSo we took the opportunity to continue discussions and progress in the development of the WAM AGL port. And, as we expected, it was quite beneficial to unblock tasks like AGL app framework security integration, and the support of AGL latest official release, Funky Flounder. Julie Kim from Igalia attended the event too, and presented an update on the progress of the Ozone Wayland port.
\nThe organization and the venue were great. Thanks to LGSVL!
\n![\"\"](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/lgsvl_logo.png\"){kind=logo}
Next trip was definitely closer. Just 90 minutes drive to our Igalia headquarters in A Coruña.
\n![\"\"](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/20170802_113339-e1540197571674.jpg\")
Igalia has been organizing this event since 2009. It is a cross-web-engine event, where engineers of Mozilla, Chromium and WebKit have been meeting yearly to do some hacking, and discuss the future of the web.
\nThis time my main interest was participating in the discussions about the effort by Igalia and Google to support Wayland natively in Chromium. I was pleased to know around 90% of the work had already landed in upstream Chromium. Great news as it will smooth integration of Chromium for embedders using Ozone Wayland, like webOS. It was also great to know the work for improving GPU performance reducing the number of copies required for painting web contents.
\n![](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/31222451768_9d85cd2760_o-e1540294341646.jpg\")
\nOther topics of my interest:
\nAs usual, the organization was great. We had 70 people in the event, and it was awesome to see all the activity in the office, and so many talented engineers in the same place. Thanks Igalia!
\n![\"\"](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/30160192427_e277b24cdc_o.jpg\")
Web Engines Hackfest 2018 CC BY-SA 2.0
\nThe last event in barely a month was my first visit to the beautiful town of Dresden (Germany).
\n![\"\"](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/20181019_150530.jpg\")
The goal was continuing the discussions for the projects Igalia is developing for AGL platform: Chromium upstream native Wayland support, and the WAM web runtime port. We also had a booth showcasing that work, but also our lightweight WebKit port WPE that was, as usual, attracting interest with its 60fps video playback performance in a Raspberry Pi 2.
\nI co-presented with Steve Lemke a talk about the automotive activities at LGSVL, taking the opportunity to update on the status of the WAM web runtime work for AGL (slides here). The project is progressing and Igalia should be landing soon the first results of the work.
\n![\"\"](\"https://blogs.igalia.com/dape/2018/10/25/3-events-in-a-month/images/20181017_094812-e1540198579290.jpg\")
It was great to meet all this people, and discuss in person the architecture proposal for the web runtime, unblocking several tasks and offering more detailed planning for next months.
\nDresden was great, and I can’t help highlighting the reception and guided tour in the Dresden Transportation Museum. Great choice by the organization. Thanks to Linux Foundation and the AGL project community!
\n\nSo… what’s next? I will be visiting San Francisco in November for Chrome Dev Summit.
\nI can only thank Igalia for sponsoring my attendance to these events. They are quite important for keeping things moving forward. But also, it is also really nice to meet friends and collaborators. Thanks Igalia!
\n\n", "date_published": "2018-10-25T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2018/03/21/updated-chromium-legacy-wayland-support/", "url": "https://blogs.igalia.com/dape/2018/03/21/updated-chromium-legacy-wayland-support/", "title": "Updated Chromium Legacy Wayland Support", "content_html": "Future Ozone Wayland backend is still not ready for shipping. So we are announcing the release of an updated Ozone Wayland backend for Chromium, based on the implementation provided by Intel. It is rebased on top of latest stable Chromium release and you can find it in my team Github. Hope you will appreciate it.
\nLinux desktop is progressively migrating to use Wayland as the display server. It is the default option in Fedora, Ubuntu and, more importantly, the next Ubuntu Long Term Support release will ship Gnome Shell Wayland display server by default (P.S. since this post was originally written, Ubuntu has delayed the Wayland adoption for LTS).
As is, now, Chromium browser for Linux desktop support is based on X11. This means it will natively interact with an X server and with its XDG extensions for displaying the contents and receiving user events. But, as said, next generation of Linux desktop will be using Wayland display servers instead of X11. How is it working? Using XWayland server, a full X11 server built on top of Wayland protocol. Ok, but that has an impact on performance. Chromium needs to communicate and paint to X11 provided buffers, and then, those buffers need to be shared with Wayland display server. And the user events will need to be proxied from the Wayland display server through the XWayland server and X11 protocol. It requires more resources: more memory, CPU, and GPU. And it adds more latency to the communication.
\nChromium supports officially several platforms (Windows, Android, Linux desktop, iOS). But it provides abstractions for porting it to other platforms.
\nThe set of abstractions is named Ozone (more info here). It allows to implement one or more platform components with the hooks for properly integrating with a platform that is in the set of officially supported targets. Among other things it provides abstractions for: * Obtaining accelerated surfaces. * Creating and obtaining windows to paint the contents. * Interacting with the desktop cursor. * Receiving user events. * Interacting with the window manager.
\nEven if Wayland was not used on Linux desktop, a bunch of embedded devices have been using Wayland for their display server for quite some time. LG has been shipping a full Wayland experience on the webOS TV products.
\nIn the last 4 years, Intel has been providing an implementation of Ozone abstractions for Wayland. It was an amazing work that allowed running Chromium browser on top of a Wayland compositor. This backend has been the de facto standard for running Chromium browser on all these Wayland-enabled embedded devices.
\nBut the development of this implementation has mostly stopped around Chromium 49 (though rebases on top of Chromium 51 and 53 have been provided).
\nSince the end of 2016, Igalia has been involved on several initiatives to allow Chromium to run natively in Wayland. Even if this work is based on the original Ozone Wayland backend by Intel, it is mostly a rewrite and adaptation to the future graphics architecture in Chromium (Viz and Mus).
\nThis is being developed in the Igalia GitHub, downstream, though it is expected to be landed upstream progressively. Hopefully, at some point in 2018, this new backend will be fully ready for shipping products with it. But we are still not there. Some major missing parts are Wayland TextInput protocol and content shell support (P.S. since this was written, both TextInput and content shell support are working now!).
More information on these posts from the authors:
\nOk, so new Wayland backend is still not ready in some cases, and the old one is unmaintained. For that reason, LG is announcing the release of an updated legacy Ozone Wayland backend. It is essentially the original Intel backend, but ported to current Chromium stable.
\nWhy? Because we want to provide a migration path to the future Ozone Wayland backend. And because we want to share this effort with other developers, willing to run Chromium in Wayland immediately, or that are still using the old backend and cannot immediately migrate to the new one.
\nWARNING If you are starting development for a product that is going to happen in 1-2 years… Very likely your best option is already migrating now to the new Ozone Wayland backend (and help with the missing bits). We will stop maintaining it ourselves once new Ozone Wayland backend lands upstream and covers all our needs.
\nWhat does this port include? * Rebased on top of Chromium m60, m61, m62 and m63. * Ported to GN. * It already includes some changes to adapt to the new Ozone Wayland refactors.
\nIt is hosted at https://github.com/lgsvl/chromium-src.
\nEnjoy it!
\n\n\n", "date_published": "2018-03-21T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2012/10/01/webkitgtk-accelerated-composition-on-wayland/", "url": "https://blogs.igalia.com/dape/2012/10/01/webkitgtk-accelerated-composition-on-wayland/", "title": "WebKitGTK+ accelerated composition on Wayland", "content_html": "Originally published at webOS Open Source Edition Blog. and licensed under Creative Commons Attribution 4.0.
\n
As part of my work at Igalia browsers team, I am working on making WebKitGTK+ and Epiphany work on Wayland.
\nJust running non 3D websites on Wayland did not involve too much work. But running the OpenGL accelerated code in WebKit was a bit more complicated. Though, I’ve got a first working version.
\n![\"Video:](\"https://blogs.igalia.com/dape/2012/10/01/webkitgtk-accelerated-composition-on-wayland/images/hqdefault.jpg\")
On WebKitGTK+, we enable the use of hardware acceleration with OpenGL for:
\nYou can read more about accelerated compositing on these posts from Martin Robinson: WebKitGTK+ hackfest wrapup, and Accelerated compositing update.
\nOn X11, we use XComposite, sharing a Window among the GTK+ widget (WebKitWebView) and the GL contexts for WebGL and accelerated composition. We have a tree of layers, each one rendering to a texture. Then these textures are composited rendering directly to the X11 window.
\nOn Wayland, things are a bit different. Wayland protocol does not define a way to share a buffer among clients, nor a way to “insert” a window inside another window. My solution is just making the accelerated compositor render the layers to another texture. When the time comes for the WebKitWebView to be drawn (using Cairo), we render this texture too. If we build GTK+ for using EGL, then this process happens completely on GPU.
\nNext step will be adding support for accelerated composition in WebKit2GTK+. The main challenge here is that the WebKitWebView widget is on UI process and the WebGL contexts and layers rendering are in Web Process. So, if we want to avoid buffers going to/from GPU, we need to share them between the two processes. DRM authentication through EGL_mesa_drm extension could help here.
\n", "date_published": "2012-10-01T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2012/07/27/lightning-talk-web-apps-stores-in-epiphany/", "url": "https://blogs.igalia.com/dape/2012/07/27/lightning-talk-web-apps-stores-in-epiphany/", "title": "Lightning talk: web apps stores in Epiphany", "content_html": "I’m just now in a train. Destination: GUADEC 2012!
\nToday I’ll be presenting a lightning talk about my work at Igalia browsers team for web application stores support in Epiphany. Take this talk as a fast follow up on the work I posted about some months ago, “Epiphany meets the web app stores”. I will also outline the plan for next months.
\nSee you at GUACEC!
\n![\"\"](\"https://blogs.igalia.com/dape/2012/07/27/lightning-talk-web-apps-stores-in-epiphany/images/going-to-guadec-2012-badge.png\"){kind=icon}
In the last few weeks, as part of my work here at Igalia, I’ve been playing a bit with the concept of hybrid applications. In this case, I’ve created a basic prototype of a mail application, with its user interface completely written using JQuery Mobile, and with backend code in C and GObject. The result is iwkmail.
\n![\"Screencast](\"https://blogs.igalia.com/dape/2012/05/14/iwkmail-mixing-webkit-gtk-camel-and-jquery-mobile/images/0.jpg\")
Though it’s a simple experiment, I’ve added some mail basic functionality, so I could try to catch as much as possible of real requirements for how we could improve the developers WebKit+GNOME experience creating hybrid applications.
\nMy first conclusion is that it’s surprisingly easy and fast to develop such applications. Second, I could reuse tons of source code and modules from my old projects. This approach surely provides a way to create cool GNOME applications, using the most fashionable web client technologies.
\nSo, you’ll get:
\nThe UI is completely written in Javascript + HTML, using JQuery Mobile.
\nThe backend side is done using Camel library inside Evolution Data Server, so we rely on a library well tested for more than 10 years. All the code related to this is implemented in C+GObject, and I reused a good set of code from Modest, the default mail client for Nokia N810 and N900. I’ve got involved on its development for 3 years, so that’s a bunch of code I know well enough.
\nFor communication, I use the AJAX-like JSONP protocol, and custom SoupRequest URI scheme handlers. Basically I expose some methods as iwk:addAcccount, iwk:getMessage, etc, and arguments are passed as usual in a web request. The result I obtain from this calls is a JSON object with the results of the call. Simple, and works very well.
\nI’ve pushed the work on github: https://github.com/jdapena/iwkmail. Feel free to try it!
\nOh, I guess it’s very obvious that I did not spend too much time thinking on the project name… So, anyone proposing something that matches the IM acronym (I don’t want to rewrite the class names!) would deserve a beer.
\nLast, lots of thanks to Igalia for giving me the opportunity to do this experiment. As usual, fun stuff to work with.
\n", "date_published": "2012-05-14T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2012/03/16/epiphany-meets-the-web-app-stores/", "url": "https://blogs.igalia.com/dape/2012/03/16/epiphany-meets-the-web-app-stores/", "title": "Epiphany meets the web app stores", "content_html": "In last weeks, I’ve been taking a look at the web applications standards support in Epiphany, as part of my work at Igalia. Xan wrote about the Save as web application feature present in Epiphany 3.2, that is a base for very simple (and userful) web applications support in Gnome desktop.
\nTo continue with this work, I’ve been investigating on adding support for some web app stores. So I’ve done an experimental implementation for Mozilla Open Web Apps (as in 2011 tech preview), Chrome Web Store hosted and packaged apps, and Chrome CRX-less apps.
\n\nScreencast using Chrome Web Store.
\nThis is an experiment. Not supported, and it may actually stay out of official Epiphany. So there are lots of things not working at all. This is first a way to have a big number of apps to play with our application mode, and improve it. So no permissions check, URL’s match may be broken, many apps will fail to even log in… Did I say it is an experiment? Most obvious issues are related with this external links handling bug.
\nBut, if you just want to play with it, just try my branch webapp in my Epiphany Github repository. By default, support is disabled, so you’ll have to enable these keys: $ gsettings set org.gnome.Epiphany.web enable-chrome-apps true $ gsettings set org.gnome.Epiphany.web enable-open-web-apps true
You can try with Mozilla Labs Apps Dir from 2011 tech preview and Chrome Web Store.
\n", "date_published": "2012-03-16T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2010/07/27/talk-about-modest-4-for-guadec-next-thursday-challenges-of-portability-between-hildon-and-gnome/", "url": "https://blogs.igalia.com/dape/2010/07/27/talk-about-modest-4-for-guadec-next-thursday-challenges-of-portability-between-hildon-and-gnome/", "title": "Talk about Modest 4 for Guadec next Thursday. Challenges of portability between Hildon and GNOME.", "content_html": "Tomorrow I’m leaving to GUADEC 2010. I’m goint to assist only on Thursday this time, when I’ll be doing this year GUADEC talk about Modest project.
\nThis time the talk focus will be completely different, as I’ll be explaining the process towards Modest 4, where we’re focusing in intensive refactoring, with the goal of releasing a product quality in GNOME, Moblin and Hildon/Maemo5 platforms.
\nAlso, I’ll talk about some differences between Maemo and GNOME platforms, and some bits I miss in GNOME platform:
\nI won’t ellaborate too much, but I’m trying to point some weak points in GNOME platform we could improve (just taking free software Maemo components, or improving GNOME platform components).
\nThe talk will be on Thursday, at 14:45 in Seville room.
\n", "date_published": "2010-07-27T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2010/04/12/clutter-grilo-player-0-1-1/", "url": "https://blogs.igalia.com/dape/2010/04/12/clutter-grilo-player-0-1-1/", "title": "Clutter Grilo Player 0.1.1", "content_html": "This week I’ve been working a bit more in the Clutter Grilo Player. And finally did release 0.1.1:
\n![\"\"](\"https://blogs.igalia.com/dape/2010/04/12/clutter-grilo-player-0-1-1/images/cgp-0.1.1-shot.png\")
Thanks to Chris Lord for his patches in clutter code, and Iago Toral for his help improving YouTube speed.
\nAs usual, I uploaded the packages to CGP Launchpad PPA. The code, in CGP gitorious.
\n", "date_published": "2010-04-12T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2010/04/05/first-release-of-clutter-grilo-player/", "url": "https://blogs.igalia.com/dape/2010/04/05/first-release-of-clutter-grilo-player/", "title": "First release of Clutter Grilo Player", "content_html": "Last weeks I’ve been playing with Clutter and Mx libraries, with the idea of knowing them deeper while also trying to help a bit. Honestly, I believe that the best way to learn about such things is just creating something using them.
\nSo, knowing the fantastic effort done by Grilo team at Igalia, to create a framework for accessing different multimedia sources across the internet, I came to the idea of creating a very simple media player that uses MX widgets, Gstreamer, and Grilo framework.
\nAnd here it is, the first release of Clutter Grilo Player. It’s still dirty, but the general idea of the interface is there. It supports browsing and searching some Grilo providers, including Youtube and others.
\n![\"\"](\"https://blogs.igalia.com/dape/2010/04/05/first-release-of-clutter-grilo-player/images/Pantallazo-Clutter-Grilo-Player.png\")
Some links:
\nAfter some migration work, now we have Tinymail repository completely migrated to gitorious.org:
\n\nI’ve rescued all the branches available in our svn and tried to keep the proper authorship attributions.
\nSo, from now on, the development should happen in gitorious, and, if you want to keep updated with the latest changes, this is the source to get the information.
\nI’ve also updated as much as possible the tinymail wiki with proper references to the gitorious.
\nI know I’ve just announced moving modest to gitorious, but modest was already in git. This time the change in tinymail is bigger as we’re also moving to git from svn! Big change, bigger benefits.
\n", "date_published": "2010-03-15T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2010/02/26/modest-mail-now-in-gitorious-org/", "url": "https://blogs.igalia.com/dape/2010/02/26/modest-mail-now-in-gitorious-org/", "title": "Modest mail, now in gitorious.org", "content_html": "This week we’ve finally moved Modest to gitorious:
\n\nThe repository itself is called modest: http://gitorious.org/modest/modest/ git://gitorious.org/modest/modest.git
\nReasons are basically that gitorious is faster and better providing git services. So I hope the change is for good.
\nAll the other services will still be in garage: mailing lists, wiki, and project web.
\nLast weeks we’ve also been writing some information about how Modest has been implemented, in the wiki. You can find them in Modest architecture documentation. There you’ll find:
\nAs I told in previous post, we’re developing actively a port of Modest for Gnome and Moblin, trying to keep the user experience we created in Maemo Fremantle releases.
\nWhile the goal is having something working for Moblin, Modest way of handling mails (kept simple, and fast to browse) is something you may definitely want to try in y our desktop.
\nSo, last weeks we’ve started to create packages of Modest for Ubuntu Karmic using the unstable development for Gnome&Moblin. For this we’re using Launchpad PPA. You can get packages for Karmic here:
\nhttps://launchpad.net/~jdapena/+archive/modest/
\nOf course, we would be glad to prepare releases for other distributions. Just ask.
\nIf you check the Modest git repository you’ll see that we’re actively developing in two branches: master and modest-3-2:
\nToday I’ve prepared release 3.90.4 of Modest. The main new feature is that it includes support for handling calendar invitation requests in plugins. So a protocol plugin would be able to handle the calendar invitations and add accept/tentative/decline buttons. Also some bugfixes.
\nIn next hours this should be available in Modest PPA.
\n", "date_published": "2010-01-24T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2009/12/11/whats-going-on-in-modest-new-gnomemoblin-port/", "url": "https://blogs.igalia.com/dape/2009/12/11/whats-going-on-in-modest-new-gnomemoblin-port/", "title": "What's going on in Modest: new Gnome / Moblin port.", "content_html": "As many of you know, Modest is the mail client of Nokia n810 and n900 devices. As that, there is a huge effort on it, to make a really good mail experience in those devices. But for last years, the effort was completely concentrated on Maemo platform.
\nLast months, Sergio Villar and me have been working on bringing the Modest user experience to both Gnome and Moblin, using our community projects time here at Igalia. The work was based on a very interesting effort from Javier Jardon this summer.
\nThe main goal was trying to make the behavior of Modest in Gnome as similar as possible to the counterpart in Fremantle/Maemo5. It’s still unstable, a work in progress, but it’s already showing how it will look like:
\n![\"List](\"https://blogs.igalia.com/dape/2009/12/11/whats-going-on-in-modest-new-gnomemoblin-port/images/Modest4GnomeFoldersView.png\")
You’ll see a really big difference with other mail clients available in desktop. It’s really oriented to keep things really simple and straightforward, so Modest is not only light, but its user experience is kept light too following a similar style to the one used in Fremantle Modest.
\nMost use cases are already functional. We’ll try to do a new release next week, and, if possible, also offer some packages for easy testing. Stay tuned.
\n", "date_published": "2009-12-11T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2009/07/09/modest-talk-at-gran-canaria-desktop-summit-slides-and-screencast/", "url": "https://blogs.igalia.com/dape/2009/07/09/modest-talk-at-gran-canaria-desktop-summit-slides-and-screencast/", "title": "Modest talk at Gran Canaria Desktop Summit. Slides and screencast", "content_html": "On monday we (Sergio and me) had the talk about Modest for Fremantle in Gran Canaria Desktop Summit. Good to see some people interested in our work.
\nJust Modest is getting a really pretty good shape. Definitely, it’s really cool what we could do with Hildon 2.2/Gtk toolkit. We moved to a really great UI in a few weeks! Also, lots of bugfixes, so Modest is far more reliable now, and also more easy to use than ever.
\nThe slides: Modest talk at Guadec/Desktop Summit 2009
\nAnd the screencast of Modest in Maemo5/Fremantle SDK
\n", "date_published": "2009-07-09T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2009/04/29/new-modest-plugin-system-anyone-willing-to-implement-rss-support/", "url": "https://blogs.igalia.com/dape/2009/04/29/new-modest-plugin-system-anyone-willing-to-implement-rss-support/", "title": "New Modest plugin system. Anyone willing to implement RSS support?", "content_html": "As Quim announced yesterday, the Beta release of Maemo5 SDK has been released.
\nIt includes our beloved Modest, which goes back to the opensource development. It’s been migrated to GIT, and you can see and track the development of the project from now on, but also browse the history of the project. See more about all of this in Modest Garage project webpage.
\nOne very interesting new feature you’ll get with Modest is the new plugin system. The protocols code has been refactored, to allow extending Modest to support new protocols, in addition to the ones supported in standard Modest (IMAP, SMTP and POP3). And this has been exposed through a plugin API.
\nWhat you can do with this is adding plugins adding support in Modest for RSS, NNTP, etc. Just imagine the kind of mail server you would like to see in Modest!
\nWe’ve added to the new created Modest development wiki some information about Modest Plugin development.
\nOne final note. This plugin API is still not assuming an API/ABI stability, but this is our goal. Bad thing is you’ll need to keep your development in sync with Modest (at least for now). Good thing is that you can still contribute or request changes to the plugin API, and this won’t be a hell.
\n", "date_published": "2009-04-29T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2008/07/24/gtk-3-0-and-beyond-team-requirements/", "url": "https://blogs.igalia.com/dape/2008/07/24/gtk-3-0-and-beyond-team-requirements/", "title": "Gtk 3.0 and beyond. Team requirements", "content_html": "The 3.0 approach of “no new functionality”, only wiping out weird stuff is good. But I have some concerns on the timing for the plan. If 3.0 is simply wiping old stuff out then, why should we wait to next Spring to finish this? Or, once we have it stabilised, why 1 more year of development to get new features? The total gap of 2 years is reeeeeeally long. Can we go faster? I see the community has the will to make Gtk better, soon, but the problem seems to be that the community doesn’t have resources for this. So, in parallel with the implementation plan for Gtk 3.0, we may think about a organization plans for 3.x or for 2010. Do we want to make Gtk grow faster, better? Is current Gtk+ core team big enough for what we need?
\nCurrently the list of core developers in Gtk+ as you can seen in the web page has 10 members. A goal would be something like this: let’s have 20 developers that deserve been in that list in 2010.
\nBut getting people trained and productive so they deserve getting a core responsibility is hard and slow. Do we want Gtk+ grow healthier, faster, safer, with more quality? Don’t we feel that the current core team members are heroes that can miraclously maintain and grow Gtk+ because they are really good doing their work? Can we help them? Any effort on growing the core team will take a while, so we should take this seriosly soon if we want results in a reasonable schedule.
\n", "date_published": "2008-07-24T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/12/17/modest-time-for-feedback/", "url": "https://blogs.igalia.com/dape/2007/12/17/modest-time-for-feedback/", "title": "Modest! Time for feedback", "content_html": "It’s been a lot of work. But Modest first beta is finally here. For Chinook/OS2008 users, it’s easily available:
\n![\"Direct](\"https://blogs.igalia.com/dape/2007/12/17/modest-time-for-feedback/images/install_button_small.png\")
You can read more details on the release in the blogs of other Modest developers: Dirk-Jan, Philip, and my workmates at Igalia Sergio, and Berto.
\nBut I want to highlight here that Modest is Beta. Not in the sense of “huh, don’t blame us if it fails”, but the opposite. Just blame us, cry, shout. We want to make sure Modest is better and better, and we just need you write good bug reports, so that we can work on them.
\n![\"Bug](\"http://commons.wikimedia.org/wiki/Image:Bug_aggregation.jpg\")
Bug reports should go to Maemo Bugzilla. Section Communications/Modest. We’re willing to know your problems, and/or enhancement requests.
\nSee you in bugzilla!
\n", "date_published": "2007-12-17T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/09/25/some-ideas-of-improvements-for-file-chooser/", "url": "https://blogs.igalia.com/dape/2007/09/25/some-ideas-of-improvements-for-file-chooser/", "title": "Some ideas of improvements for file chooser", "content_html": "I’m not saying that file chooser is bad at all for selecting files, but there are two features I would definitely love in it:
\nThe use case I found where I need this is “adding attachments in evolution”. If I want to send a mail, I usually want to avoid sending very big files as this is not very good for most mail accounts people use.
\nIf I want to do this with Evolution/Gnome, I have to open the folder with Nautilus to check the size of the files.
\nIn fact, you can find other interesting information you’ll want to check when you are opening a file with file chooser. For example, the id3 tags of a mp3/ogg file.
\nI suppose the UI design experts can point good ideas about this. For me it would be enough if the contextual menu of a file would offer a “Properties” action.
\nWe don’t need thumbnailing only for opening a file from gimp or to attach an image. The thumbnail of a image works better than the file name to describe the contents of a file, so we should enable the user to use it to know which file he’s managing, even when the application is not a imaging related one. So I suppose it should enable thumbnails by default.
\nJust provide a way to open in nautilus a folder you’re viewing in file chooser. Not sure how, as a “open folder” button would lead to confussion. Maybe “browse here”? This way you could easily access to all the operations available in nautilus for files and folder, without having to browse to a folder two times (one in file chooser and one in nautilus).
\n", "date_published": "2007-09-25T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/09/19/purging-attachments-with-tinymail-or-being-smart-with-small-storages/", "url": "https://blogs.igalia.com/dape/2007/09/19/purging-attachments-with-tinymail-or-being-smart-with-small-storages/", "title": "Purging attachments with Tinymail (or being smart with small storages)", "content_html": "Imagine you have a pretty mobile or portable device with email access. You love using it for reading mail just because you don’t have to use your laptop or pc, and your fantastic device can go in your pocket or handbag.
\nThis device is fantastic, yes, but it has a limited storage capability. And you usually get a lot of mails, and some of them really big (with images, audio files or even videos). It’s not too difficult to exhaust the storage in some days or even hours.
\nI added an API to Tinymail that lets you remove attachments stored locally (remove from cache in case the message is from a remote folder, or remove permanently if the folder is stored locally). The API methods are these:
\n![\"AGL](\"https://blogs.igalia.com/dape/img/logo_agl_horizontal_white.svg\"){kind=logo}
\ntny_mime_part_set_purged (TnyMimePart *part): this method marks a mime part to be purged.tny_msg_rewrite_cache (TnyMsg *msg): this method rewrites the message to local storage (cache or not), but removing the attachments marked as purged.Then, if you want to remove some attachments from a message, you only have to mark them as purged using the set_purged method, and then, make the change persistent with the rewrite_cache method.
Currently this is only available for IMAP and local MAILDIR provided folders, but it shouldn’t be too difficult to add support for this to other providers.
\nThe inners of the purge method are really simple. The only thing set_purge method does is setting the Disposition field of the mime part with the value purged. In camel backend:
camel_mime_part_set_disposition (priv->part, \"purged\");This mark is mostly innocent, and the implementation is the same for any mime part, independently of the provider (IMAP, maildir or whatever).
\nThen rewrite_cache simply rewrites all the message to the local representation of the message, but ignoring the contents of the mime parts marked as purged this way. What we get this way is a reduction in the local storage (we don’t expunge the header of the mime part, only the contents). We also keep more or less the same structure of the message, as the original mime part headers are still stored, even when they get empty contents.
The Purge API gives you more freedom to choose what you want to keep in your storage. You can not only wipe out full messages, but also wipe out specific attachments you don’t want to keep.
\n", "date_published": "2007-09-19T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/03/01/gnome-buildbot-released-at-fosdem/", "url": "https://blogs.igalia.com/dape/2007/03/01/gnome-buildbot-released-at-fosdem/", "title": "Gnome buildbot released at FOSDEM!", "content_html": "Yes, as expected, Iago Toral published the work in Gnome Buildbot at FOSDEM. You can read his announcement in the blog.
\nBriefly, if you maintain a project in Gnome, and want automatic tests running, then Gnome Buildbot may interest you. And, if you have a project built using jhbuild, then it’s easy to create a Jhbuild based buildbot as the one for Gnome. Just ask in #build-brigade.
\n", "date_published": "2007-03-01T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/02/23/tomorrow-talk-about-build-brigade-in-fosdem/", "url": "https://blogs.igalia.com/dape/2007/02/23/tomorrow-talk-about-build-brigade-in-fosdem/", "title": "Tomorrow, talk about Build Brigade in FOSDEM", "content_html": "This Saturday, Iago Toral will be presenting Build Brigade at FOSDEM. He’ll be presenting some work that we’ve been doing last months, including the Gnome Buildbot. More information in FOSDEM web and in his blog:
\n\nI talked previously about Gnome Buildbot. Briefly, it’s a Continuous Integration based in Buildbot service that uses jhbuild to compile the full Gnome moduleset. It runs tests and returns coverage reports for many modules.
\n", "date_published": "2007-02-23T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/02/17/wikipatterns/", "url": "https://blogs.igalia.com/dape/2007/02/17/wikipatterns/", "title": "WikiPatterns", "content_html": "Via Agile testing blog I knew about Wikipatterns, a repository of patterns and antipatterns about wiki adoption and usage. Very interesting work.
\nI would highlight some interesting entries:
\nIn Igalia we’ve been using wikis+mailing lists for internal communication for a long time. In Gnome we’ve got Live wiki. And many other communities have found wikis a powerful resource for communication and documentation. Good reading for all these groups.
\n", "date_published": "2007-02-17T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/02/06/integration-of-unit-tests-in-a-gnome-buildbot/", "url": "https://blogs.igalia.com/dape/2007/02/06/integration-of-unit-tests-in-a-gnome-buildbot/", "title": "Integration of unit tests in a Gnome Buildbot", "content_html": "Today I’ve read about the great work of Iago Toral (a workmate at Igalia Innovation) with Gnome Buildbot. He’s been doing some efforts to integrate unit tests of Gtk+ in the build loop.
\nUsing check, he’s generated wonderful HTML reports, and a nice integration with the standard buildbot compilation view:
\n![\"Gtk+](\"https://blogs.igalia.com/dape/2007/02/06/integration-of-unit-tests-in-a-gnome-buildbot/images/testsdetail.png\")
You can read more information about Iago’s work in his blog entry “Gnome buildbot and integration of unit tests”. Hopefully we’ll get a public deployment of Gnome Buildbot soon.
\n", "date_published": "2007-02-06T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/01/29/snes-emulators-in-touchscreens-loving-igalia-and-lovely-events/", "url": "https://blogs.igalia.com/dape/2007/01/29/snes-emulators-in-touchscreens-loving-igalia-and-lovely-events/", "title": "Snes emulators in touchscreens, loving Igalia and lovely events", "content_html": "As many of you know, I am a worker, and one of the initial partners of Igalia. This company is organized following an assembleary model. Every partner has the same rights and power of decission. And we want all workers to become partners in a medium term.
\nThe main representation of this assembleary model is the Assembly itself. It’s a periodic meeting (one each two months) with all the partners discussing and taking the strategic choices of the company. We’ve held 43 assemblies in the history of Igalia, the first one in September 2001.
\nNo bosses at all, just all of us being partners, with the same responsibility. This is fair, and one of the better things of being a member of Igalia team. And this makes everybody contribute with their best efforts and ideas.That’s one of the main reasons I love working here. And other main reason may be how much extraordinary and good people my partners are.
\nThis is a very interesting event that will happen this year. It stands for Guadec + aKademy = Guademy destroy myths. As the event web states, Guademy is a meeting where GNOME and KDE developers share working sessions with other people interested in collaborating with both projects. It will be in March in A Coruña (Galicia - Spain), and is being organized by GPUL. Good idea and best wishes!
\nLast week I’ve been playing a bit with some emulators, in order to try to get one of them running in Maemo. The one I’ve been testing more deeply is Snes9x. It compiles without too much problems, and it should be easy to make it run a confortable UI with some hildon work.
\nBut there’s a problem. Nokia N800 and 770 haven’t got too many hardware buttons, and a good Super NES emulator should get 6 fire buttons plus start and select buttons. My idea would be add support for showing buttons in the screen replacing some hard buttons in the screen, in order to use the touch screen for providing 4 fire buttons. Unfortunately, I suppose the main drawback of this idea is that AFAIK the touchscreen of these devices can’t handle more than one “touch” simultaneously :(. Any idea about this?
\nThe other problem will probably be hardware performance, but I didn’t test the emulator in the device yet.
\n", "date_published": "2007-01-29T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2007/01/16/new-year-new-life-and-new-city/", "url": "https://blogs.igalia.com/dape/2007/01/16/new-year-new-life-and-new-city/", "title": "New year, new life (and new city)", "content_html": "Long time since last post. Lots of news had happened, so I will not bore you with lots of them. Anyway, some interesting remarks:
\nLast month I’ve moved to a flat in Vigo with my girlfriend. Lovely city, pretty flat and very nice to be living with her! We’ve been very busy with the move and preparing our wedding. It seems next months won’t be more relaxed…
\n", "date_published": "2007-01-16T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/11/17/maemo-bugzilla-and-first-build-brigade-irc-meeting/", "url": "https://blogs.igalia.com/dape/2006/11/17/maemo-bugzilla-and-first-build-brigade-irc-meeting/", "title": "Maemo bugzilla, and first build brigade IRC meeting", "content_html": "You don’t realize how wonderful Gnome Bugzilla is till you find some other that needs a bit of love.
\nAs I’ve told in my last post, I’m hacking a bit in Maemo platform, and in particular in Sardine Distro. I found some issues I wanted to report, and implemented some patches for them. So, I decided to go to Maemo Bugzilla. But the experience was far from optimal. I’ve sent two mails to maemo-developers mailing list (problems with categorization, suggesting a bug template).
\nWe need to decide what to do in Build brigade next month. Iago Toral has proposed a meeting for next Monday at 16:00 UTC, in #build-brigade IRC. Everyone interested in Gnome testing or continuous integration, is invited. I hope we’ll get at least one meeting per month, but let’s talk this Monday about this. You can read more about the meeting in Iago’s blog and meeting proposal in mailing list. See you there!
\n", "date_published": "2006-11-17T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/11/10/playing-with-maemo/", "url": "https://blogs.igalia.com/dape/2006/11/10/playing-with-maemo/", "title": "Playing with Maemo", "content_html": "Long time without writing, so let’s retake it. Last weeks I’ve been playing with Maemo and Nokia 770. Very interesting platform, very familiar for those (like me) with some background in Gnome SDK.
\nThe device is fantastic. Definitely. Small, light, and with a good screen. But overall, as much powerful as free software is. As this was my main interest, Maemo as a free software based device, I’ve been taking a look at the development platform. Nokia has done a good work, not only providing full access to a Gtk/Gnome based technology, for developing applications, but also offering their platform contributions to the community.
\nInteresting points:
\nDrawbacks:
\nOf course, I’ve sent my first bugs and patches mainly related with file selection (filters bug, proposal for filters UI). I’ll comment more interesting things I find next days.
\n", "date_published": "2006-11-10T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/10/06/developer-testing-is-good-for-you/", "url": "https://blogs.igalia.com/dape/2006/10/06/developer-testing-is-good-for-you/", "title": "Developer! Testing is good for you.", "content_html": "Once Build brigade puts in production the Buildbots for projects, with testing support, we’ll get an automated way to run unit tests of all projects in Gnome umbrella. And you, developer, will love to know when your tests fail to get them fixed asap.
\nBut, to do this, you should implement good tests. How to do these tests? A good unit testing framework is Check.
\nIf you need an example: currently, Iago Toral is developing unit tests for gtk+ with it. You can browse his work or read his post to know about them or ask him at #build-brigade. It’s an interesting work, and accepting and extending these unit tests in mainstream gtk+ would be great for gtk+ quality.
\nTwo main goals in your tests:
\nWouldn’t it be great that you know your new features don’t break other code behaviour before committing? Just run make check and you’ll begin to be more confident about that.
Of course, these notes apply directly only to C projects. But replace Check with your favourite test framework for your language ;).
\n", "date_published": "2006-10-06T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/10/03/first-jhbuild-buildbot-prototypes-online/", "url": "https://blogs.igalia.com/dape/2006/10/03/first-jhbuild-buildbot-prototypes-online/", "title": "First jhbuild buildbot prototypes online", "content_html": "Yesterday I’ve deployed my jhbuild buildbot prototypes:
\nCode coverage measure consists on counting the portion of code being executed by a set of tests. It gives a quantitative measure of the paths of code covered by our tests, and it’s specially useful for unit testing. One of the purposes of Gnome Build Brigade is adding this kind of measures to the continuous integration loop. This way, we can encourage developers to increase their marks, or even make parallel teams add new tests to the core libraries of Gnome. This would make Gnome more reliable, I think.
\nThen, if you want to polish your library or project, please add some pretty tests. If you want help on Check, just contact us in #build-brigade gnome irc channel. Better quality in Gnome deserves this!
\nPS: it seems the link to Iago Toral’s test is not working (Bonsai seems to get in troubles when a module has some symbols like +). You can get the source of the tests in our CVS (CVSROOT :pserver:anonymous@cvs.igalia.com:/var/publiccvs MODULE gtk+).
This week I’ve been adding some new features to my JHBuild Buildbot scripts, in order to get better reporting and information for projects:
\nIn order to get a smaller testing platform for my jhbuild buildbot scripts, I’ve decided to target on Fisterra project. To do this, I had to work a bit on it in order to make it easy to compile in a JHBuild environment:
\nAnd yes, now I can run Fisterra in a JHBuild environment. The work has been uploaded to the CVS (bonsai of fisterra-jhbuild). I’ve uploaded instructions in Fisterra CVS web. The available modules from the scripts are:
\nYes! Today, 21-September-2006, Igalia celebrates its 5th anniversary!
\n![\"nad0001_001sized.jpg\"](\"https://blogs.igalia.com/dape/2006/09/21/igalia-celebrates-5th-anniversary/images/nad0001_001sized.jpg\")
An example of party at Igalia office (year 2003 ;))
\n", "date_published": "2006-09-21T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/09/19/jhbuild-buildbot-integration-scripts-done/", "url": "https://blogs.igalia.com/dape/2006/09/19/jhbuild-buildbot-integration-scripts-done/", "title": "JHBuild Buildbot integration scripts... done!", "content_html": "Yesterday I’ve ended the documentation and upload of my JHbuild buildbot integration scripts. I’ve published a manual (jhbuild buildbot manual). With this bot, I’ve got:
\nmake check for all Gnome modules supporting this Makefile rule, and then it runs lcov to explore the code coverage of those tests.This week I’ll try to get this server in production (currently I’m running it in my PC). There’s a CVS with the scripts, and you can browse them with our Bonsai.
\nThese days I’ve also been preparing some patches for Fisterra framework and apps to get them easily compiled in JHBuild. I’ll try to merge the patches this week. They fix compilation with gcc 4, and make it easier to get the modules compiled in a non-standard prefix (as JHBuild requires).
\nNext thursday Igalia will be 5 years old. Fine to be such a long time with these guys :).
\nLast saturday, I’ve gone with my girlfriend to an interesting restaurant in Vigo (Rosalia de Castro street). Tasty food (in special the hot chocolate fritters, or the veal tenderloin with strawberries and cheese sauce).
\n", "date_published": "2006-09-19T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/09/08/gcov-tests-with-gcc-4-x/", "url": "https://blogs.igalia.com/dape/2006/09/08/gcov-tests-with-gcc-4-x/", "title": "GCov tests with gcc 4.x", "content_html": "Thanks to Thomas for giving me this clue to get coverage reports with code compiled with gcc 4.x.
\nIn order to get coverage logs, CFLAGS should include -fprofile-arcs -ftest-coverage (and preferrable to get also -O0). It’s what the gcc manual says. It was enough for gcc 3.x. But in gcc 4.x it may not. When you link, you can get this error:
hidden symbol `__gcov_init' in /usr/lib/gcc/i486-linux-gnu/4.0.3/libgcov.a(_gcov.o) is referenced by DSOThe problem seems to be caused by the linker. In theory, adding -ftest-coverage should imply linking gcov automatically. But it’s not true, at least in my case. The solution: adding -lgcov to the LDFLAGS. Finally I have coverage reports for my gcc 4.1.2 compiler.
CFLAGS=-fprofile-arcs -ftest-coverage
LDFLAGS=-lgcovFirst, hello to planet.gnome.org! Thanks to jdub for making it possible. Briefly, I’m working at igalia as a member of the Gnome technologies team, and currently I’m devoted to continuous integration and testing tasks under the umbrella of the Build brigade group. Hackergotchie is comming up, stay tuned.
\nBack from holidays. Some notes:
\nBut now I’m back. These days I’ll try to go on with the efforts around Buildbot and jhbuild integration:
\nThis week I’ve been playing a bit with buildbot, in order to complete my hack to get gnome jhbuild integrated with it. And it’s running now. I’ve uploaded a patch to buildbot project webpage (buildbot jhbuild support patch in sf).
\nThe ideas of the patch are the following:
\nI added an example of a configuration file for buildbot master to get this working. Now I’m running gnome jhbuild in my machine. Saddenly the UI of buildbot does not scale very well for large sets of modules :(.
\nA capture here:
\n![\"jhbuild-buildbot-capture.jpeg\"](\"https://blogs.igalia.com/dape/2006/07/28/jhbuild-buildbot/images/jhbuild-buildbot-capture.jpeg\")
Today I’ve found a problem trying to create a patch with cvs. I haven’t got write access to gnome repository. This way I have only read-only access to it. The problem is when I try to create a patch with new files involved.
\nIf I run:
\n$ cvs diff -Nit ignores the new files. If I run:
\n$ cvs diff -N changedfile1 changedfile2 newfile1 newfile2it says something like this for every new file:
\ncvs server: I know nothing about newfile1An easy solution would be creating two patches:
\n/dev/null.But there’s another trick to get a standard patch from cvs. You have to edit the CVS/Entries file of the directory you want to add a file. For example, for a newfile1 in directory directory1, you would change the directory1/CVS/Entries file and add an entry like this one:
/newfile1/0/Initial newfile1//\nThen I can do a standard cvs diff command like this one:
\n$ cvs diff -Nand this will include the new files in the patch. Of course you can use the -u/-U parameters to get unified format patches.
Since the last post, I’ve began to implement all the required methods to provide a complete DBus functionality in jhbuild. I added the following DBus interfaces:
\nYou can hook to the following signals:
\nThe biggest problem I’ve faced these days was related to the way I could launch the subprocess for build/update commands. As it’s called in a dbus handler, I couldn’t put the jhbuild script in another thread. And the method should return immediately without waiting for the end of the command. I had to use the gobject.Idle interface to hook the build to the gobject mainloop, this way:
gobject.Idle child, and add a callback implementation calling the builder script and returning False (in order to be called only one time).The code for the idle child is something like this:
\nclass JHBuildBuilderIdle(gobject.Idle):
def __init__(self, builder):
gobject.Idle.__init__(self)
self.set_callback(self.callback, builder)
def callback(self, builder):
builder.build()
return FalseAnd the call in the dbus handler is this:
\n@dbus.service.method('org.gnome.JHBuildIFace')
def build(self, modulelist = [], skiplist = [], start = None, datespec = None, options = {}):
[...]
build = jhbuild.frontends.get_buildscript(ownconfig, module_list)
idle = JHBuildBuilderIdle(build)
idle.attach()
returnTomorrow I’ll add this work to the jhbuild bugzilla in order to begin the discussion upstream.
\n", "date_published": "2006-07-17T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/07/13/dbus-support-for-jhbuild/", "url": "https://blogs.igalia.com/dape/2006/07/13/dbus-support-for-jhbuild/", "title": "DBus support for JHBuild", "content_html": "Today I’ve began the experiment to add DBus support to JHBuild. To do this I’ve established these goals:
\nAnd guys, I did it! I’ve added a new dbus command to jhbuild to launch it as a DBus service, and begin to wait for user calls.And added a dbus frontend to buildscript. This frontend offers signals to know the status of a compilation (offering methods to get the logs, and to know the current status of compilation.
\nIf I run:
\ndape@bonus:~$ dbus-send --print-reply --dest='org.gnome.JHBuild'
/org/gnome/JHBuildObject org.gnome.JHBuildIFace.buildit launches a complete build. For example, I can use dbus-monitor to view events, and I get a log like this one:
signal sender=:1.95 -> dest=(null destination) interface=org.gnome.JHBuildIFace; member=end_phase_signal
string \"libxml2\" string \"checkout\" string \"false\"
signal sender=:1.95 -> dest=(null destination) interface=org.gnome.JHBuildIFace; member=start_phase_signal
string \"libxml2\" string \"build\"
signal sender=:1.95 -> dest=(null destination) interface=org.gnome.JHBuildIFace; member=message_signal
string \"Building libxml2\"
signal sender=:1.95 -> dest=(null destination) interface=org.gnome.JHBuildIFace; member=message_signal
string \"make all-recursive make[1]:
se ingresa al directorio `/usr/local/devel/dape/cvs/libxml2'
Making all in includeThis way we can communicate with the JHBuild compilation loop. It should be interesting to avoid running jhbuild (and loading python) for each compilation in an integration loop. But it also adds an asynchronous channel to communicate with continuous integration tools. I’ll go on completing the dbus interface for most used commands of jhbuild tomorrow, and also add customizability (currently it does not wait for many kinds of parameters, it should be improved). When it’s more polished I’ll propose the patch for jhbuild.
\n", "date_published": "2006-07-13T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/07/12/adapting-jhbuild-to-continuous-integration/", "url": "https://blogs.igalia.com/dape/2006/07/12/adapting-jhbuild-to-continuous-integration/", "title": "Adapting JHBuild to continuous integration", "content_html": "These days I’ve been doing some work in JHBuild to make life easier for those who want to run it from a Continuous Integration loop. In particular, it should improve the experience for those using Buildbot (as we intend in the Gnome Build Brigade).
\nIn Buildbot, most used build scripts enable you to decide how checkouts are done. The options are currently four:
\nFor me, the most interesting options are the copy mode and the clobber mode. In continuous integration loops, we usually don’t add modifications over the compilation trees, but merge conflicts raise frequently. With those options, you get an easy solution.
\nI’ve added a bug and a patch to JHBuild bugzilla to support checkout options. With this patch, you can establish a global checkout mode, and specific checkout modes for each module in jhbuildrc file.
\nCurrently JHBuild lets you run make check for modules using the autotools module type. If you set the makecheck variable in jhbuildrc, it will run make check before installing the packages. The problem with this is: if the checks fail, then JHBuild considers the module is broken, and then it will not compile the modules depending on it.
I’ve written a patch to change this behavior. With this, you can set up a unittestfails variable (currently defaulting to True to maintain current behavior) in jhbuildrc. If this variable is false, make check failures don’t break compilations of the module. It goes on compiling, but shows a message in the log warning about this.
There was an old bug talking about this in JHBuild bugzilla. I added the patch to this bug.
\nTwo ideas for JHBuild integration with other tools:
\nI’ll be working on this next week.
\n", "date_published": "2006-07-12T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/07/04/guadec-2006-experience-build-brigade/", "url": "https://blogs.igalia.com/dape/2006/07/04/guadec-2006-experience-build-brigade/", "title": "Guadec 2006 experience. Build brigade!", "content_html": "As I wrote last week, I’ve been attending to Guadec 2006 in Vilanova i la Geltru. A very positive experience, with lots of meetings, talks and interesting events in general.
\nFor last thursday, there was a BOF about Continuous Integration hosted by Juan José Sánchez. There, I presented the work with Tinderbox here in Igalia, and we discussed a bit about the requirements for a Continuos Integration service in Gnome project. The interest was more than I expected, and we shared our experiences and decided to create a work group to get the infrastructure up. It’s called Gnome Build Brigade, and we’ve set up a wiki (yes, all belongs to wiki!) Currently we have the fantastic work from Frederic Peters (JHAutobuild). It’s running last months, and offers RSS information about compilation status for developers.
\nBut Thomas Vander Stichele proposed Buildbot, as they’re using it in Fluendo for gstreamer. Main reasons: well maintained, big community and more mature. So it seems the way to go, and I began to test it yesterday, as I’m planning to move Fisterra continuous integration to Buildbot.
\nYes, buildbot is very easy. It’s more strict about clients (slaves in its notation), but it lets server administrator get a better control over them. And the model is more secure. I’ve also been learning a bit about Twisted, the framework it uses to implement an asynchronous events system. I also implemented a prototype of integration with jhbuild that is working for small sets of modules, but does not handle dependencies yet. My compromise with Build brigade group is about jhbuild work anyway. I’ll be these days working on JHBuild integration, to extend it to fit better with CI systems (I’ve prepared a patch for stages splitting, and I would want to add more options to checkout). I’ll write about these next days.
\n", "date_published": "2006-07-04T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/06/25/guadec-2006/", "url": "https://blogs.igalia.com/dape/2006/06/25/guadec-2006/", "title": "Guadec 2006", "content_html": "Yes, I’ve got to Guadec this Friday. After a long trip (fast flight, not so fast train trip), we’ve arrived to the Gnome Village. There you can find lots of Gnomers from nearly any project you’re interested in.
\nVilanova i la Geltru is a quiet village near Barcelona. Unfortunately the Gnome Village is not near the Guadec rooms, so there’s been lots of problems to move people among the places (dining rooms, centre of the town, bungallows in the village). Also it seems there are few taxis, so if you do something wrong with bus schedules, you’re driving in heavy problems.
\nBut it’s Guadec. Lot’s of interesting presentations. Now I’m writing from the Carpa and listening to an interesting speech by John Laerum about how to do a good presentation. I’ve realized I’m not very good at my presentations, but the tips are interesting, and I hope I learn something for my own presentations in the future.
\nMeanwhile I’m going on improving the Gnome Tinderbox 3 deployment at Igalia. Hope it can be complete enough this Thursday, so we can talk about this in the Continuous Integration BOF here in Guadec.
\n", "date_published": "2006-06-25T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/06/21/tinderbox-3-building-gnome/", "url": "https://blogs.igalia.com/dape/2006/06/21/tinderbox-3-building-gnome/", "title": "Tinderbox 3 building Gnome", "content_html": "Here in Igalia we’re very interested in some aspects of software quality, and in special in continuous integration. We’re very used to have continuous integration services for our projects, as one of the fundamentals to be handle a group of programmers accessing the same repository.
\nFor doing this, we’ve used three different continuous integration servers, depending on the project:
\nAs a result of the petition in Gnome Love list for a tinderbox we started an effort to adapt Tinderbox to Jhbuild, and run modulesets inside. It led to two parallel works, one with Tinderbox2 and one with Tinderbox3. Meanwhile, BxLUG has done some work in the same direction, which you can view in the Gnome JhAutobuild webpage.
\nThe Tinderbox2 work was done easily. You can check the current experimental status of the portal in our Tinderbox2 based Gnome Tinderbox. We are integrating unit tests and coverage in it, and we’ve also added RSS feeds for the modules (see here).
\nAnd now I’m working in a Tinderbox 3 setup. As the T2 one, it’s experimental. You can check it in our Gnome Tinderbox3 webpage. I’m tweaking and hacking this days, so it’s changing fast. Some features are:
\nI would like to share this effort with community. Related to this, a BOF will be held in GUADEC 2006 next week in Vilanova. There’s more information about the continuous integration BOF in GUADEC webpage. It will be on next Thursday 29 at 12:00. See you there!
\n", "date_published": "2006-06-21T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/06/06/my-first-programming-book/", "url": "https://blogs.igalia.com/dape/2006/06/06/my-first-programming-book/", "title": "My first programming book", "content_html": "Yesterday I saw an entry in Juanjo’s blog talking about the book he started to learn programming with. And immediately I decided I had to look for my own start book.
\nIt was when I was 7 years old, and my parents gave us an MSX computer (a Toshiba HX-10). One month later my father decided he wanted to implement software for his business (he’s got a clinic laboratory), and bought a very big book, MSX, Guía del programador y manual de referencia (The Complete MSX Programmers Guide, by T.Sato, P.Mapstone and I.Muriel, 1984).
\n![\"Front](\"https://blogs.igalia.com/dape/2006/06/06/my-first-programming-book/images/msx-guia-programador.jpg\")
It was the beginning. Since them my life was heavily tied to computer science. I’ve played many games in computers, written a lot of documents, pictures or even music. But the start was a BASIC program. This year I’ve done 20 years programming.
\n", "date_published": "2006-06-06T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/04/25/let-s-kill-the-templates-in-gtk-doc/", "url": "https://blogs.igalia.com/dape/2006/04/25/let-s-kill-the-templates-in-gtk-doc/", "title": "Let's kill the templates in gtk-doc", "content_html": "Once upon a time, there was gtk-doc. It was weird and strange, and overall, different from other standards like javadoc or doxygen. But we gnomish people, were tied to it, as it was the only one working correctly with GtkObject/GObject. It’s still this way, as I don’t know any good solution with Doxygen or other tool to fill the space of gtk-doc.
\nWhy it was weird? For me, the worst point was the need to maintain a set of files called templates. They were docbook templates that covered some documentation that you couldn’t add to your source code. These files were processed by some scripts that didn’t work very well with version control systems.
\nWith gtk-doc 1.5 we can avoid the templates, at least for most of the use cases they were required. Now you can add to the source code a commentary like this one:
\n/**
* SECTION:myclassname
* @short_description: This is my short description for #MyClassName
* @see_also: #Other, #AnyOtherOne
*
* It's a long description about this and that, and other things. You can write
* about many things and enter sgml tags like:
* <itemizedlist><listitem>an itemized list</listitem><itemizedlist>
*/With this structure you remove at last the templates requirement. One additional note. As there was a bug in gtk-doc 1.5, the general makefile scripts provided don’t work with no templates. You have to add a rule to the documentation Makefile.am like this:
tmpl/*.sgml: scan-build.stampAnother interesting point is using gtk-doc.make. It’s a script provided by the gtk-doc distribution providing general structure for gtk-doc processing. You can use it adding this line to the Makefile.am:
include $(top_srcdir)/gtk-doc.makeFor all of you folks that always complain about how difficult it is to create latex documents with those complex Makefiles compiling figures, linking eps’s, doing bad works to generate pdf or ps automatically, latex-make is here.
\nIt’s a makefile include file to help you compile and track dependencies of a latex document. For example, if you want to compile a latex-only document (of one or more files), you only have to create a Makefile with one line. Imagine the master tex file is called mydocument.tex. Then you would write a Makefile with two lines like this:
LU_MASTERS=mydocument.tex
include LaTeX.mkIf you want to state which kind of outputs you want it to compile by default, you only have to state the flavors (PS, DVIPDF and/or PDF):
\nFLAVORS=DVIPDFAnd about the figures. If you have the figures of your document in figures directory, you only have to add:
LU_mydocument_GPATH=figuresThere are more interesting points. It adds a module to handle .fig files and bibliographies.
This week I’ve been hacking a while around Dogtail. It’s a Python library intended for implementation of functional tests of applications providing accessibility support through AT-SPI. It lets you check and manipulate the user interface from Python scripts, and it’s easy to integrate it into test cases.
\nWhile I was working, I found that I could not easily browse tabs in a GtkNotebook using dogtail. This issue has been reported in Gnome bugzilla (bug report). The problem is:
\nI’m not the only one that has detected the problem this way. Peter Johanson has sent two bugs and patches to solve the issue and add tab browsing support in Dogtail (report in Pyspi and report in Dogtail). In parallel I was developing similar patches. As Peter Johansons’ are more complete, I’ve simply added some functionalities this morning to the dogtail patch to provide an easier way to use the selection interface in Dogtail. I’ve uploaded the new patch to the Dogtail bug report.
\nNow you can simply write:
\nfocus.application('gedit')
focus.widget(roleName='page tab', name='My tab')
focus.widget.node.select()And it raises the desired tab easily. Hope those patches go upstream. I think they’re a good solution for the tab browsing issue.
\n", "date_published": "2006-03-30T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/03/20/dbus-and-the-power-and-network-managers/", "url": "https://blogs.igalia.com/dape/2006/03/20/dbus-and-the-power-and-network-managers/", "title": "DBUS and the power and network managers", "content_html": "Some time ago, in the Guadec-es in Coruña, there was an interesting meeting to talk about interesting enhancement in Gnome desktop. It was mainly centered in i18n issues, but I talked about the way other platforms work in network APIs.
\nBriefly, I missed a way to know about network status and manage connections, as other platforms do (for example, we get this kind of API in Microsoft Windows related SDK with Connection Manager API. The same applies to power management.
\nBut now there’s an interesting work to face this issue. If Network manager and Power manager DBUS message providers get more or less standarised in Linux distros, we’ll get an standard API to do this kind of use cases:
\nIt seems these managers, and their use of DBUS, are the way to go. Good luck!
\n", "date_published": "2006-03-20T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/02/13/bed-exams-and-rock-n-roll/", "url": "https://blogs.igalia.com/dape/2006/02/13/bed-exams-and-rock-n-roll/", "title": "Bed, exams and rock'n'roll", "content_html": "Long time since last time I’ve written here. I’ve been sick some days, trying to work in some stuff. I’ve got my midterm exams in Escuela Oficial de Idiomas (State Language School). I’m doing the 5th degree of English (the last one) here in Vigo. Seems they were good exams. The bad point was the writing exam, which was based on a Lost episode I haven’t seen (7th of the second season). Fortunately it seems it didn’t contain any spoilers.
\nLocal music
\nLast weeks I went to two concerts of local groups. The first was in El Ensanche (Vigo), and was performed by Iván Ferreiro, with the collaboration of Xoel. They performed as Rai Doriva e as Ferreiro, playing classic songs as Inbetween days by The Cure, some songs from Andrés Calamaro and so on. Good to see them again.
\nAnd yesterday I went to a concert of Heissel in Pub Aturuxo (Bueu). Second time I’ve seen them, and again, interesting. This time we talked a bit with some members of the group. They’ve just released their first recording El lugar más feliz(Happiest place).
\nDebian Gnome
\nLast week Gnome 2.12 has finally hit completely Debian Sid. Big improvement Rhythmbox to my music player. Libnotify popups, queued-to-play songs UI management and other things. Congratulations to developers!
\n", "date_published": "2006-02-13T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/01/27/request-for-help-in-debian-gnome-team/", "url": "https://blogs.igalia.com/dape/2006/01/27/request-for-help-in-debian-gnome-team/", "title": "Request for help in Debian GNOME team", "content_html": "In this week release of Debian Weekly News there’s a request for help from Debian GNOME team. You can see a list of required tasks from Alioth.
\nMaybe an oportunity for you folks, if you want to help in such an important task.
\n", "date_published": "2006-01-27T00:00:00Z" } , { "id": "https://blogs.igalia.com/dape/2006/01/26/welcome/", "url": "https://blogs.igalia.com/dape/2006/01/26/welcome/", "title": "Welcome!", "content_html": "Hi guys. Here in Igalia many of us are starting to blog. So welcome to this area!
\n", "date_published": "2006-01-26T00:00:00Z" } ] }