Presenting the Web Engines Hackfest

Posted in Blink, English, Planet, WebKit on October 28th, 2014 by Manuel Rego Casasnovas – Be the first to comment

After the Google’s fork back in April 2013, WebKit and Blink communities have been working independently, however patches often move from one project to another. In addition, a fair amount of the source code continues to be similar. Thus, it seems interesting to have a common place to discuss topics of shared interest and make plans for the future.

For that reason, Igalia is announcing the Web Engines Hackfest that would happen in December 7-10 in our main office in A Coruña (Spain). This is like a new edition of the WebKitGTK+ Hackfest that we’ve been organizing since 2009, but this year changing the focus in order to make a more inclusive event.

The Hackfest

The new event will include members from all parts of the Web Platform community, not restricted to WebKit and Blink, but open to the different Web (Gecko and Servo) and JavaScript (V8, JavaScriptCore and SpiderMoney) free software engines.

Past year hackfest picture by Claudio Saavedra

Past year hackfest picture by Claudio Saavedra

The idea is to bring together implementors from the Open Web Platform community in a 4-day hacking/discussion/unconference event with the goal of moving forward different topics, discuss common issues, make plans for the future, etc.

And that’s what makes this event something different. It’s a completely hacking oriented event, where developers sit together in small groups to work for a few days pursuing some specific goals. These groups of interest are defined at the beginning of the hackfest and they agree the different tasks to be done on during the event.
Thereby, don’t expect a full scheduling in advance with a list of talks and speakers like in a regular conference, this is something totally different focused on real hacking.

Join us

After the first round of invitations we’ve already a nice list of people from different companies that will be attending the hackfest this year.
The good news is that we still have room for a few more people, so if you’re interested in coming to the event please contact us.

Adobe and Igalia are sponsoring the Web Engines Hackfest 2014

Adobe and Igalia are sponsoring the Web Engines Hackfest 2014

Thanks to Adobe and Igalia for sponsoring the Web Engines Hackfest and make possible such an exciting event.

Looking forward to meet you there!

CSS Grid Layout Automatic Placement

Posted in Blink, CSS, CSS Grid Layout, English, Planet, WebKit on June 19th, 2014 by Manuel Rego Casasnovas – 1 Comment

In his last post my mate Sergio explained the different syntax to position elements inside a grid. Now is time to talk about the automatic placement feature, how it works and show some examples of its potential.

Auto-placement

The concept is defined in the specification:

Grid items that aren’t explicitly placed are automatically placed into an unoccupied space in the grid container.

So, let’s start with a simple example to show how it works. Imagine that you have the following 2×2 grid:

<div style="display: grid;">
    <div style="grid-row: 1; grid-column: 1 / span 2;">first</div>
    <div style="grid-row: 2; grid-column: 2;">second</div>
    <div>auto</div>
</div>

first item is using the 2 columns on the 1st row, second item is placed on the 2nd row 2nd column. That leaves an empty space on the 2nd row 1st column, where the auto item is placed.

Simple example where the auto-positioned item is placed into the 2nd row 1st column

Simple example where the auto-positioned item is placed into the 2nd row 1st column

Of course you can do more complex things, like setting a specific position in one dimension (row or column) and left the other as auto, request more than one cell for the item, etc. as you can see in the following example of a 3×3 grid:

<div style="display: grid;">
    <div style="grid-column: span 2;">item 1</div>
    <div style="grid-column: 3;">item 2</div>
    <div style="grid-row: span 2;">item 3</div>
    <div style="grid-row: span 2; grid-column: span 2;">item 4</div>
    <div style="grid-row: 2;">item 5</div>
</div>
Example animation of how different auto-positioned items are placed in the grid

Example animation of how different auto-positioned items are placed in the grid

Let’s analyze why elements are positioned as you can see in the animation above:

  • item 1: This is the first element placed, this item need 2 columns and it’s placed in the 1st row 1st-2nd columns, as they’re empty.
  • item 2: This item is only specifying the column, so it’s placed in the 1st row 3rd column, as it’s the first empty cell in the 3rd column.
  • item 3: In this case the items needs 2 rows and it’s placed in the 2nd-3rd rows and 1st column.
  • item 4: Here the example needs 2 rows and 2 columns, we’re lucky as there’re some empty cells so it’s placed in the 2nd-3rd rows and 2nd-3rd columns.
  • item 5: This item needs to be placed in the 2nd row, as there’s no empty cells in that row, it’ll be placed in a new column outside current grid, 2nd row 4th column. Imagine that this item is completely auto (we remove the “grid-row: 2;” style), then it’d be placed in a new row outside current grid, 4th row 1st column.
    Why in a new row and not in a new column? That’s managed by grid-auto-flow property that’ll be explained below.

grid-auto-flow property

Again from the spec:

The grid-auto-flow property controls the direction in which the search for unoccupied space takes place, and whether rows or columns are added as needed to accommodate the content.

The simpler (and easier to understand) values for this property are row (by default) and column. These values define the direction that the auto-placement algorithm should follow to place the auto-positioned items. For example if the value is row the algorithm will try to fill each row before jumping to the next one, adding new rows as necessary.

Also, this property allows to combine these values (row and column) with other keywords (that cannot not be used together):

  • dense: By default auto-placement algorithm is sparse, that means that if placing an auto item has leave some empty cells in the grid (because of the item doesn’t fit) they won’t be used anymore. If dense is used, these holes will be used if smaller items come up later.
  • stack: In this case the auto-placement algorithm looks for the first empty cell (like if it was adding a 1×1 item). Then it places all the auto-positioned items in that cell (independently of their size), stacked atop one another.

As usual it’s easier to understand with a HTML example:

<div style="display: grid;">
    <div style="grid-row: 1; grid-column: 2;">item 1</div>
    <div style="grid-column: span 2;">item 2</div>
    <div>item 3</div>
</div>
Example grid with different values for grid-auto-flow property

Example grid with different values for grid-auto-flow property

In the image you can check how the auto-positioned elements (item 2 and item 3) are placed depending on the different values of grid-auto-flow:

  • sparse (grid-auto-flow: row;):
    • item 2: It looks for an empty area of 2 columns, so it skips cell in 1st row 1st column levaing it as an empty space. The item is placed in the 2nd row 1st-2nd columns.
    • item 3: It looks for the next empty space, and it adds the item in the 3rd row 1st column. This will leave a hole in the 1st row 1st column.
  • dense (grid-auto-flow: row dense;):
    • item 2: This is placed exactly the same than in the sparse case. The item is placed in the 2nd row 1st-2nd columns.
    • item 3: It looks for an empty cell and it adds the item in the 1st row and 1st column as it’s still empty (item 2 didn’t use it before, because of it doesn’t fit there).
  • stack (grid-auto-flow: stack row;):
    • item 2: It looks for the first empty cell, and it finds the 1st row and 1st column. Then all the auto-positioned items will be placed there. So item 2 is placed in 1st row 1st-2nd column (even if it takes more than one cell, it’ll keep using as reference the first empty cell).
    • item 3: Again this is added in the 1st row 1st column over item 2.

Possibilities

As you can imagine this brings a lot of power to the CSS Grid Layout spec. You won’t need to always place your items explicitly in a grid, you can simply pass the responsibility to find the best position to the grid itself.

The typical example is to think in a regular form:

<form>
    <label>Name</label><input type="text" />
    <label>Mail</label><input type="text" />
    <label>Comments</label><textarea></textarea>
    <label>Accept policy</label><input type="checkbox" />
    <div id="buttons">
        <button>Accept</button>
        <button>Cancel</button>
    </div>
</form>

You could just apply the following CSS and you’d have a formatted form “for free”:

    form     { display: grid;           }
    label    { grid-column: 1;          }
    input    { grid-column: 2;          }
    #buttons { grid-column: 1 / span 2; }

The result will be something like the following picture:

Formatting regular form using grid auto-placement feature

Formatting regular form using grid auto-placement feature

Wrap-up

We’ve created a small demo (as part of our grid examples repository) that will allow you to play with the auto-placement feature and understand it better:

http://igalia.github.io/css-grid-layout/autoplacement.html

As you could read during the post, the auto-placement algorithm has been mentioned several times. This algorithm and the implementations details will be explained in a subsequent post.

If you’re wondering about current support in Blink and WebKit, the basis of auto-placement feature is already working on both engines. In Blink span support in auto-positioned items has recently landed and in WebKit the patch is ready to be integrated.
However, the new grid-auto-flow syntax (sparse, dense and stack) is still on the way (follow the Blink and WebKit bugs if you’re interested).
As you can see, we’ve been working hard on implementing the missing bits of this feature and we hope it would be fully supported soon in both Blink and WebKit. :-)

Finally, we’d like to thank again Bloomberg for sponsoring our work around CSS Grid Layout.

Igalia & Bloomberg logos

Igalia and Bloomberg working together to build a better web

CSS Grid Layout at BlinkOn 2

Posted in Blink, CSS, CSS Grid Layout, English, Planet on May 19th, 2014 by Manuel Rego Casasnovas – 1 Comment
Zurich Google Office

Zurich Google Office

Past week I’d been attending BlinkOn 2 at Google office in Zurich. I think that the best part of this kind of events is to have the chance to meet many well known people that you only see usually in the bug tracker and IRC.

It was nice to see other companies around too like Opera, Samsung, Intel, etc. We were four igalians there talking about the work we’re lately doing on Blink (check Sam’s post for more information).

CSS Grid Layout

As you might expect, this was my primary focus in the conference, as that’s what I’ve been working on during the last months.

Me during the CSS Grid Layout talk (photo by Calvaris)

Me during the CSS Grid Layout talk (photo by Calvaris)

On Tuesday, I was giving a talk about the work we’re doing on CSS Grid Layout sponsored by Bloomberg. It seems that people are interested in this feature and they liked the presentation, you can check the slides.

Julien Chaffraix (grid’s reviewer) was there and later we discussed about the approach to fix some issues that have been hanging around in the bugzilla for a while.

Selection

On Wednesday, I attended the selection talk given by Yoshifumi Inoue and Hayato Ito. They’re now focused on fixing the issues in Shadow DOM with selection.

Actually, they have some interesting ideas regarding how to avoid the problems with DOM ranges. Basically, they’ll try to use a start and end points for the selection instead of range itself, and then they’ll iterate over the composed tree in the right order.

On top of that, we talked about the issues that are present with other layout models, but we just concluded that selection is very complex and we should try to solve things step by step.

Thanks to the editing team for arranging the session. :-)

Panel discussion in the main room (Tech Talk)

Panel discussion in the main room (Tech Talk)

Other

Many other interesting talks were given during the event, like the 60fps talk by Nat Duca, the multi-column sessions by Opera guys, the Oilpan presentation by Mads Ager and many more. If you missed any of them you can check the slides in the BlinkOn 2 web page and see the videos of the talks in the main room whenever they’re available online.

Finally, I’d like to thank Igalia for sponsoring my trip and, of course, Google for the event (specially Max for the great organization and also Philip for arranging the hiking on Thursday which, despite of the foggy weather, was pretty nice).

Zurich view from Felsenegg

Zurich view from Felsenegg

Selection interaction in CSS Regions is now spec compliant

Posted in CSS, CSS Regions, English, Planet, WebKit on May 5th, 2014 by Manuel Rego Casasnovas – Be the first to comment

This post is a recap to summarize all the work we’ve been doing in Igalia during the last months related to selection in CSS Regions.

Wrap-up

Back in October 2013 my mate Javi explained the different problems we detected on how selection worked in pages using CSS Regions. As part of this initial work, we were improving test coverage as it was explained in a previous post. Originally we were working both in WebKit and Blink, but as you probably know CSS Regions has been removed from Blink code during this year.

In parallel, we’ve been also working in optimizations over CSS Regions code. First providing some new selection performance tests and later implementing some performance patches in the part related with re-paint phase, which is very important during the selection process.

Making selection spec compliant

Our first solution was put aside as it had some rough edges and it was probably too ambitious as initial step.

Nonetheless in my last post we introduced a new idea to make selection in CSS Regions spec compliant. We called it the “Subtrees approach“. We’ve finally implemented it and the patch is already integrated upstream.

Example of selection in CSS Regions before and after the Subtrees approach patch

Example of selection in CSS Regions before and after the Subtrees approach patch

You can check the new behaviour using our test suite. As you could see now selection content always matches with the highlighted text in the web page.

Future

Somehow this is just the tip of the iceberg, a small step to make selection in CSS Regions work better. However, selection is still not completely natural from the user point of view in several cases. On top of that, selection in other layout models (flexbox, grid, multi-column, etc.) has very similar issues.

Our plan is to start a discussion inside WebKit and Blink communities about how selection works in the different layout models and what kind of changes can be done there to create a better user experience around selection interaction on the web (as Javi has introduced in his last post).

Finally, we’d like to publicly thank Adobe Web Platform team for giving us the chance to collaborate with them, specially Mihnea for all his support. Also, thanks to all the reviewers for their insightful feedback and precious time.

Welcome CSS Grid Layout

Posted in Blink, CSS, CSS Grid Layout, English, Planet, WebKit on March 13th, 2014 by Manuel Rego Casasnovas – 3 Comments

Igalia has been working in the implementation of CSS Grid Layout since past year (more below). This year I’ve had the chance to join the team and help to move forward this great spec.

State of the Art

Maybe, you’ve never heard before about CSS Grid Layout, but for sure you know part of the history of design in the web platform.

  • In the beginning people tried to make cool designs using <table> tag. Which completely breaks the separation between content (HTML) and presentation (CSS), apart from the ugly code and poor results achieved with this approach.
  • Then people started to use <div> for everything, floating blocks here and there. This might cause a terrible mess when trying to do flexible layouts that will adapt themselves to the screen size.
  • Now lots of CSS frameworks are emerging (960 Grid, Blueprint, Bootstrap, Foundation, Skeleton, …). They allow to create responsive designs (very useful in the mobile era) most of them based on the grid concept. The problem is that each framework has its own syntax and lots of CSS code in order to provide the expected behavior.

To solve all these issues W3C has defined a new powerful and flexible spec that will make web designer’s life happier.

CSS Grid Layout

“allows authors to easily define complex, responsive 2-dimensional layouts”

By Tab Atkins Jr. (Google) at CSS WG Blog.

CSS Grid Layout provides a mechanism to divide the available space in rows and columns with a set of predictable sizing behaviors. It defines a set of grid areas where designers can precisely place the elements of a web page.

CSS Grid Layout example

CSS Grid Layout example

The Grid Layout spec can be used to intelligently reflow elements within a web page optimizing locations and sizes, depending on the device where the page is rendered in, creating responsive designs.

Example

So, let’s imagine that you have the following HTML and you want to layout it like in the picture above:

<div class="grid">
    <div class="title">Title</div>
    <div class="menu">Menu</div>
    <div class="main">Main</div>
    <div class="footer">Footer</div>
</div>

This will be the CSS syntax required to do the magic:

.grid {
    display: grid;
    grid-template-columns: 200px 1fr;
    grid-template-rows: 100px 1fr auto;
}
.title  { grid-column: 1; grid-row: 1;          }
.menu   { grid-column: 1; grid-row: 2 / span 2; }
.main   { grid-column: 2; grid-row: 1 / span 2; }
.footer { grid-column: 2; grid-row: 3;          }

Where:

  • display: grid: Defines a grid container.
  • grid-template-columns and grid-template-rows: Specify the track breadths.
  • grid-column and grid-row: Determine a grid item’s size and location within the grid.

Flexible tracks (the ones defined with 1fr in the previous example) will grow or shrink and automatically adapt themselves to the viewport size. This allows to create very customized grids, defining flexible breadths depending on the contents or the available space.

We’ve created a repository with some examples and demos, feel free to play with them to get a better understanding of the spec capabilities:

http://igalia.github.io/css-grid-layout/

Implementation status

Despite of being a Working Draft (WD) spec, implementations are appearing in the main web engines:

  • IE/Trident first shipped an implementation in IE10.
  • Google started an implementation in WebKit by the end of 2011. Igalia has been working on this implementation since mid-2013, collaborating with Google in Chromium/Blink and maintaining an up-to-date implementation in Safari/WebKit.
  • Mozilla started working on an implementation in Firefox/Gecko this year.

Acknowledgements

Finally, we’d like to thank Bloomberg for sponsoring our work around CSS Grid Layout on Blink and Webkit, together with the implementation of ECMAScript 6 (ES6) features in V8 and SpiderMonkey JavaScript engines.

Igalia & Bloomberg logos

Igalia and Bloomberg working together to build a better web

As a related note, my mates Juanjo and Xavi were talking about CSS Grid Layout and CSS Regions in the last Mobile World Congress at Barcelona as part of the W3C booth. Check the slides in case you missed it (demos and videos of this post are extracted from them).

Igalia will keep working hard to evolve the implementation of CSS Grid Layout spec both in Blink and WebKit. We’ll try to keep you informed about our progress, stay tuned!

Performance of selection with CSS Regions in WebKit and Blink (Part II – perf profiler)

Posted in Blink, CSS, CSS Regions, English, Planet, WebKit on January 29th, 2014 by Manuel Rego Casasnovas – 2 Comments

After the initial post introducing this topic and describing the Performance Tests (perftests), now is time to explain how to analyze the performance issues with a profiler in order to improve the code.

“Manual” measurements

First of all, you can think of doing some measurements in the source code trying to find the possible bottle necks in an application. For example you could use the following lines inside WebKit/Blink in order to measure the time required to execute a given function:

timespec ts;
clock_gettime(CLOCK_REALTIME, &ts);
printf("%lld.%.9ld\n", (long long)ts.tv_sec, ts.tv_nsec);
// Call to the function you want to measure.
timespec ts2;
clock_gettime(CLOCK_REALTIME, &ts2);
printf("%lld.%.9ld\n", (long long)ts2.tv_sec, ts2.tv_nsec);
printf("Diff: %lld.%.9ld\n", (long long)ts2.tv_sec - ts.tv_sec, ts2.tv_nsec - ts.tv_nsec);

Actually this is a pretty bad idea for a number of reasons:

  • Checking directly times with printf() is wrong as you are adding I/O that will spoil the measurements.
  • You have to modify the source code for every statement you want to measure, this can be really hard in large projects like WebKit/Blink.
  • There are better tools out there, called profilers, explicitly designed for this very same purpose, let’s use them.

Using perf profiler

In this case we are going to talk about how to use perf in a GNU/Linux environment to analyze the performance of the WebKitGTK+ port. You could follow the next steps:

  • Install it. It depends on your distro, but it should be simple. For example in Debian: apt-get install linux-tools
  • Run perf record to get the data. This will create a file called perf.data. Here you have different options:
    • Call directly the application, it will follow the children processes, so it works properly in WebKit2 architecture with multiple processes: perf record -g <app>
    • Connect perf to an already existing process, for example to the WebProcess: perf record -p <process-pid> -g
  • Use perf report to analyze the data gathered by perf record. Simply use the following command (where -i <file-name> is optional as by default it reads perf.data file): perf report -i <file-name>

About how to collect the data, in WebKitGTK+ you have the alternative to generate the perf data files while running the perftests. Just adding some arguments to run-perf-tests script: Tools/Scripts/run-perf-tests --platform=gtk --debug --profile

Which will create a file called test.data under WebKitBuild/Debug/layout-test-results/ folder.

Analyze profile session

perf report provides you the list of methods that have been running for more time, with the percentage of how many time was spent in each of them. Then you can get the full backtraces from the different places the method is called and know how many times it was invoked from each trace.

Let’s use a concrete example to illustrate it. This is a perf report output got from the WebProcess doing a big selection in a page with CSS Regions:

- 6.26% lt-WebKitWebPro libwebkit2gtk-3.0.so.25.5.0 [.] WebCore::LayoutUnit::rawValue() const
- WebCore::LayoutUnit::rawValue() const
- 33.10% WebCore::operator+(WebCore::LayoutUnit const&, WebCore::LayoutUnit const&)
- 39.54% WebCore::LayoutRect::maxX() const
- 83.75% WebCore::LayoutRect::intersect(WebCore::LayoutRect const&)
- 98.69% WebCore::RenderRegion::repaintFlowThreadContentRectangle(WebCore::LayoutRect const&, bool, WebCore::LayoutRect const&, WebCore::LayoutRect const&, WebCore::LayoutPoint const&) const
WebCore::RenderRegion::repaintFlowThreadContent(WebCore::LayoutRect const&, bool) const
WebCore::RenderFlowThread::repaintRectangleInRegions(WebCore::LayoutRect const&, bool) const
- WebCore::RenderObject::repaintUsingContainer(WebCore::RenderLayerModelObject const*, WebCore::IntRect const&, bool) const
- 75.49% WebCore::RenderSelectionInfo::repaint()
WebCore::RenderView::setSelection(WebCore::RenderObject*, int, WebCore::RenderObject*, int, WebCore::RenderView::SelectionRepaintMode)

As you can see, 6.26% of the time is spent in LayoutUnit::rawValue() method, this is used in lots of places. 33.10% of the time it’s called from WebCore::operator+() which is also quite generic. We should keep going deeper in the call-graph till we reach some methods that are interesting in our particular case.

In this case, the selection starts in RenderView::setSelection(), so we should investigate further the methods called from there. Of course, in order to do that you need to have some understanding of the code where you’re moving or you’ll end up completely lost.

Improve source code

Thanks to this data I realized that in each RenderSelectionInfo::repaint() it’s used the RenderObject::containerForRepaint(). Which most times returns the parent RenderNamedFlowThread for all its children in the render tree.

This causes that for every element under RenderNamedFlowThread the method RenderFlowThread::repaintRectangleInRegions() is called. Taking a look to this method it has a loop over all the regions forcing a repaint. This means that if you have 1000 regions, even if you’re just selecting in one of them, a repaint in the rest of regions is executed.

So, I’ve provided a patch that repaints only the affected regions, it means around 12%, 18% and 73% improvement in Layout/RegionsSelection.html, Layout/RegionsSelectAllMixedContent.html and Layout/RegionsExtendingSelectionMixedContent.html perftest respectively.

Doing tests with more regions using this example we got the following results:

Regions Without patch (ms) With patch (ms) Improvement
100 923 338 63%
150 2712 727 73%
200 5952 1285 78%
500 81731 7868 90%

As expected, results are better as we increase the number of regions.

Conclusions

This was just one specific example in order to explain how to use the available tools, trying to provide the required context to understand them properly.

For sure, there’s still plenty of work to be done in order to improve the performance of selection with CSS Regions. Nonetheless, we still have to settle a final implementation for the selection in CSS Regions before going on with the optimization efforts, as it was explained in my previous post we’re on the way to fix it.

This work has been done inside the collaboration between Adobe and Igalia around CSS Regions.

In Igalia we have a great experience in performance optimization for CSS standards like CSS Flexible Box Layout, CSS Grid Layout and CSS Regions. Please don’t hesitate to contact us if you need some help around these topics.

Approach to fix selection with CSS Regions (WebKitGTK+ hackfest late wrap-up)

Posted in Blink, CSS, CSS Regions, English, Planet, WebKit on January 23rd, 2014 by Manuel Rego Casasnovas – 1 Comment

As you probably know if you have been following this blog, we’ve been working for a while in selection interaction with CSS Regions inside a collaboration between Adobe and Igalia.

First of all, let’s contextualize this post.

WebKitGTK+ Hackfest 2013

Back in December it was organized the 5th WebKitGTK+ Hackfest at Igalia offices in A Coruña (Spain). During that week Mihnea, Javi and myself were working together trying to find a good solution for selection in CSS Regions.

As Javi has already explained in his last post, our initial implementation to fix the issues of selection in documents with CSS Regions was discarded due to several problems.

At that point, the main goal was to find an approach that will make selection in CSS Regions spec compliant and might be validated by the editing experts in order to get it integrated upstream.

Current problem

According to editing spec selection is DOM based. If you select some stuff in a page with CSS Regions, the content is retrieved properly from the DOM tree. In a selection retrieved content and highlighted content should match. However, this is not happening with CSS Regions.

CSS Regions introduces a divergence between DOM and render trees. In CSS Regions some nodes of the DOM tree (called content nodes, the ones that have -flow-into property) are moved from the “regular” render tree (under RenderHTML) to the RenderFlowThread (a sibling of RenderHTML).

During the selection we have a start and end positions in the DOM tree that are mapped into positions in the render tree. Selection algorithms traverse the render tree from start to end, which has some issues with CSS Regions in several situations. Let’s use the following diagram to explain some of them.

Diagram showing DOM and render trees together with visual display of a CSS Regions example

Diagram showing DOM and render trees together with visual display of a CSS Regions example

Let’s imagine we perform a selection from content-1 to content-2.

  • DOM selection: Returns the following nodes: content-1, source-1 and content-2.
  • Render tree selection: We start and end the selection outside the flow thread. However, source-1 which is selected in the DOM tree is under the RenderFlowThread and it’s not processed by the algorithm while traversing the render tree from start to end. Thus, it’s not highlighted.
  • Issue: source-1 is not highlighted.
Example of current and expected behaviors for selection in CSS Regions

Example of current and expected behaviors for selection in CSS Regions

Ideas

After some discussion and lots of diagrams representing the DOM and render trees on paper, we came up with 2 ideas:

  • Subtrees approach: The idea is to split the render tree in subtrees calculating the start and end positions of the selection for each of them. Then it will use current selection methods to perform the selection inside each subtree.
  • DOM order approach: In this case we would use DOM order to traverse the render tree. Also when looking for the container of a node, we will follow the DOM tree too if it’s a content node.
Handwritten diagrams representing DOM and render trees

Handwritten diagrams representing DOM and render trees

During the hackfest we implemented 2 quick prototypes to check if previous ideas would solve the problems. In both cases results were satisfactory.

However, DOM order idea seems more complicated as some elements in the DOM tree might not be present in the render tree. On top of that, we should review carefully the different methods involved in the selection and modify how they traverse the tree (or look for their container) in some cases. This would imply several changes in the more complex parts of the selection code.

On a first sight the subtrees approach is more promising as it looks simpler and cleaner. After talking with Ryosuke Niwa and David Hyatt on IRC it was confirmed as the preferred approach and likely valid to be integrated into WebKit. In addition, it seems that more performance optimizations could be done with this approach. So, let’s explain deeply how this solution would work.

Subtrees approach

All selection algorithms are thought to work in a single render tree under RenderView. Actually, RenderHTML is the top element of the render tree where selection state is set (see RenderBoxModelObject::setSelectionState()).

Inside this render tree the start object is always before than the end object. But that’s not always true in the case of CSS Regions, as you can see in the diagram above. For example, if you start selection inside the source-1 element (depending on its position inside the DOM tree) start position could be under the RenderFlowThread and end under the RenderHTML.

So, this solution will split the render tree in subtrees where the root objects would be:

  • On one hand, the RenderHTML (for the “regular” render tree).
  • On the other hand, all the RenderFlowThreads (there will be 1 per flow thread in the page).

Then the total number of subtrees we’ll be 1 (the RenderHTML subtree) + number of flow threads.

For each subtree we’ll calculate the start and end positions (start will be always before end). And then we’ll call selection algorithms for each subtree using that positions.

In the previous example we’ll have 2 subtrees:

  • RenderHTML subtree.
  • RenderFlowThread (flow-1) subtree.

Let’s calculate start and end positions for each subtree taking into account the example above:

  • RenderHTML subtree:
    • start: content-1, offset: start offset set by user’s selection.
    • end: content-2, offset: end offset set by user’s selection.
  • RenderFlowThread (flow-1) subtree:
    • start: source-1, offset: start of source-1 object.
    • end: source-1, offset: end of source-1 object.
Subtrees approach DOM and render trees selection example

Subtrees approach DOM and render trees selection example

Using these subtrees RenderView::setSelection() method will be called for each of them. Subtree’s root object will need to store information about start and end positions (together with their offsets).

Conclusion

This is an important milestone for the work we have been doing lately, and it allows us to have a clear path to make progress on this field and eventually fixing the selection in CSS Regions.

Of course, there’s still quite a lot of work to do. We’ll improve current prototype to convert it in a real patch to be uploaded at WebKit’s bugzilla and got it integrated after the review iterations. Then, as usual, we’ll try to get integrated into Blink too.

Finally, thanks to The GNOME Foundation and Igalia for sponsoring the WebKitGTK+ Hackfest, and also to Mihnea for attending and helping us to move this topic forward sharing his experience on the CSS Regions standard.

Exciting times ahead, stay tuned! Happy hacking. :-)

Sponsored by GNOME Foundation logo

Igalia logo

Performance of selection with CSS Regions in WebKit and Blink (Part I – Perftests)

Posted in Blink, CSS, CSS Regions, English, Planet, WebKit on December 10th, 2013 by Manuel Rego Casasnovas – 2 Comments

In a nutshell, CSS Regions are a new W3C specification that allows next generation responsive design in the web platform (you can read this article for a nice introduction). Adobe WebPlatform team is doing a great job leading the implementation of this standard and we at Igalia are joining efforts to contribute to the challenge.

Selection’s performance plays an important role in the collaboration between Adobe and Igalia around CSS Regions. This is the first of a series of articles on how to fix performance issues in WebKit/Blink using the selection in regions use case as example.

Example of selection performance in a document with and without CSS Regions in WebKitGTK+

Example of selection performance in a document with and without CSS Regions in WebKitGTK+

We’ve been checking the performance of selection in regular pages compared to pages using CSS Regions. We created some examples (that you can try by yourselves) and they clearly show that the selection with regions is pretty slower as you can see in the picture. On the one hand this was expected due to complexity of CSS Regions, on the other hand we wanted to take a deeper look and see how it can be improved.

First of all we need objective numbers in order to show that the new patches improve (or not) the performance. For this purpose WebKit and Blink provide a mechanism called Performance Tests (perftests). Perftests measure the time required to run them and also the memory they consume. The idea is to create a baseline with these values and spot performance regressions in new revisions (see some nice charts generated from them).

On a low level note, perftests are simply HTML files which generally use a JavaScript object called PerfTestRunner which provides different utility functions, you can find them under PerformanceTests/ folder (both in WebKit and Blink). In addition you can run them using Tools/Scripts/run-perf-tests script.

Running Layout/RegionsSelection.html perftest directly in WebKitGTK+ MiniBrowser

Running Layout/RegionsSelection.html perftest directly in WebKitGTK+ MiniBrowser

There were already some CSS Regions perftests (see PerfTestRunner/Layout/ directory) but none of them about selection. So we decided to create a new perftest for selection in CSS Regions. The new test has 100 regions and it selects text from the first region to the last one passing through the rest of regions. That way we’re somehow simulating the interaction of a selection done by a real user. The test is called RegionsSelection.html and it’s available in WebKit and Blink.

Note: CSS Regions perftests are still skipped by default because of the fact that regions performance is an ongoing development. They’ll be enabled in the future once it’s possible to create a baseline.

Just to let you know how it looks like when you use the script, below you can see the results of running the perftest in WebKitGTK+ (using r160082):
$ Tools/Scripts/run-perf-tests --platform=gtk -2 Layout/RegionsSelection.html
Running 1 tests
Running Layout/RegionsSelection.html (1 of 1)
DESCRIPTION: Testing selection with 100 regions. Select text from first region to last one passing through all the regions.
RESULT Layout: RegionsSelection: Time= 987.760100001 ms
median= 990.745000017 ms, stdev= 8.95593764346 ms, min= 978.406000009 ms, max= 1013.26800004 ms
RESULT Layout: RegionsSelection: JSHeap= 173468.6 bytes
median= 173957.0 bytes, stdev= 1973.74275506 bytes, min= 169707.0 bytes, max= 174869.0 bytes
RESULT Layout: RegionsSelection: Malloc= 1593350.4 bytes
median= 1591104.0 bytes, stdev= 15450.5740939 bytes, min= 1567544.0 bytes, max= 1622624.0 bytes
Finished: 31.423325 s

Finally, once we have a perftest that provides objective numbers, we can start using a profiler and extract some data that helps us modify the code in order to improve the performance. We’ll talk about that in the next post. Stay tuned if you don’t want to miss what Igalia is doing related to CSS Regions.

Improving tests coverage for selection with CSS Regions in WebKit and Blink

Posted in Blink, CSS, CSS Regions, English, Planet, WebKit on November 6th, 2013 by Manuel Rego Casasnovas – 1 Comment

It’s been almost a year since my previous post in this blog, I think this deserves an explanation. This year I’ve been working intensively in web internals as part of Igalia’s browsers team. Besides other stuff, I’ve been working in WebKitGTK+ port and its new WebKit2 API, contributing some patches into Epiphany (GNOME Web Browser) and lately working in the CSS Regions implementation in WebKit (where I’m already a committer :-D) and Blink as part of the collaboration between Adobe and Igalia.

Inside this collaboration, our main goal is to fix the selection issues in documents using CSS Regions. You can read more information about it on Javi Fernández’s post. As explained there we have cooked some examples highlighting different issues with selection in CSS Regions.

Despite of having more than 35,000 tests in WebKit, there were almost no tests checking the selection in CSS Regions. Thus, we’ve been creating different layout tests that you can find under LayoutTests/fast/regions/selection/ folder in WebKit‘s and Blink‘s source trees.

Example of layout test for selection in CSS Regions using vertical-rl writing mode

Example of layout test for selection in CSS Regions using vertical-rl writing mode

Most of them are reftests, which means that we have two HTML files that should render the very same result. Specifically in this case we usually compare the result of the selection content and highlighted text in CSS Regions with the result of a crafted HTML page using absolute positions. Moreover, we try to do the tests in a way that you can check them manually directly opening the HTML file a browser. Besides, we’ve created tests for different writing modes like horizontal-bt, vertical-lr and vertical-rl.

At this point we have a good tests coverage that should be enough as base to prevent breakages in the selection with CSS Regions. Anyway, it’s clear that more tests will come in the future as soon as implementation advances and some issues get fixed. You can follow the work we’ve been doing on this meta-bug.

Just for the record, here you have the full list of WebKit bugs and Blink issues related to this task:

Finally, I’d like to thank Igalia for giving me the chance to get involved in browsers development, working in the implementation of last specifications is a challenging but very heartwarming task. Let’s keep working hard to improve the web experience!

LibrePlan: Adapt planning according to timesheets

Posted in English, Java, LibrePlan, Planet on November 28th, 2012 by Manuel Rego Casasnovas – Be the first to comment

During the last weeks I’ve been working in a new feature for LibrePlan that allows to update the project plan using the data from the timesheets.

Let’s use an example in order to explain the new functionality better.

Imagine that you have scheduled a task to be done during week 45, however the timesheets reporting hours to that task are in week 47. This means that you’re not fulfilling the plan and you have a delay of 2 weeks in that task. Besides, maybe that task has dependencies with other tasks, so this delay also means that other tasks are going to be delayed.

Some questions:

  • How could you check that information in LibrePlan?
  • How could you update the current plan?

In order to solve these problems a new feature has been developed. The final implementation includes different things which will be explained below.

  • Mark a task as finished from the timesheets

    In the timesheets UI (both in standard and personal timesheets) you’ll have the option to mark if a task is finished or not. You can only mark a task as finished once, but the task end date will be taken from the last timesheet reporting hours over that task. Then, when you use the new button to adapt the planning, a new 100% progress of type timesheets will be set for the task, which means that the task has been finished.

    Pop-up to mark task as finished in personal timesheets

    Pop-up to mark task as finished in personal timesheets

  • New marks in reported hours bar

    If you use the button Show/Hide reported hours, you’ll see the pink bars in the Gantt representing the hours reported in the timesheets compared with the planned (or estimated) hours. Now, together with that information you’ll also see two square brackets, that represent the first and last date in which there’re reported hours in the timesheets.

    Gantt showing marks with information from timesheets

    Gantt showing marks with information from timesheets

    As you can see in the picture most of the tasks started more or less when it was planned. However, we can detect some issues:

    • Draft task is delayed 2 weeks, which makes Template task to be delayed too.
    • Shopping cart task was started in the planned date, however it’s taking more time than expected.
  • Adapt planning button

    There is a new button (close to Reassign) in the project Gantt view that will allow you to adapt your current planning with the information extracted from the timesheets. This will allow you to modify your planning as people report hours in the timesheets.

    Planning adapted according timesheets data

    Planning adapted according timesheets data

    Once the planning has been adapted according to timesheets data, you’ll see the actual status of your project. Moreover, you’ll have the possibility to know which tasks are finished. In the example the finished tasks are only: Draft, Product management and Shopping cart.

    With all this information you’ll be able to modify the future planning in order to finish the project in the expected date (maybe allocating more resources, reducing some tasks, …).

Summarizing, we think that this is a nice addition that could be helpful to our users in order to manage better their projects. It’ll be available in the upcoming version 1.3.2 (you can already test it in the unstable environment) and, as usual, we’ll be really glad to receive your feedback about it. If you want to know more, you can review the analysis and implementation notes at LibrePlan wiki.

Finally, we’d like to publicly thanks to SOMABE for sponsoring this development. As you probably know Igalia offers this kind of customizations and other services around LibrePLan, don’t hesitate to contact us.