Search

Ziran Sun’s blog

– Just another Igalia Blogs site –

Igalia CSR

WASH in Schools

Madina Tindano is an elementary school student who lives in Bogandé, East Burkina Faso. For a long time, the toilets in her school remained unusable and completely abandoned. Now in her final year at the school (CM2), Madina is overjoyed with the project of renovation of the latrines in her school. Thanks to the promotion of children’s right to education through better access to water, sanitation and hygiene (WASH) project.

The NGO behind the WASH Project is the UNICEF Foundation in Spain. As one of their educational projects, WASH believes that every child has the right to a quality education including access to drinking water, sanitation and hygiene services while at school. This can impact students’ learning, health, and dignity, particularly for girls like Madina . “When I see my period, I am very embarrassed to come to school because we don’t have toilets.” Madina said. WASH aims to improve access to water, sanitation and hygiene in 12 rural schools, including Madina’s, in the East region of Burkina Faso. 2796 students, 77 teachers and 48 parents will benefit from this project.

Igalia has been collaborating with UNICEF Foundation in Spain since 2007 in promoting access to quality education for children in Africa, and is proud to have been a part of this effort by contributing the funds to help 2 rural schools, which will help 430 students, 11 teachers and 7 parents. To make sure the project goes as planned, a monitoring commission is formed including some UNICEF members and representatives from Igalia (Javier Fernández and María Piñeiro).

The project started in February 2022 and was expected to finish in a year’s time. Unfortunately, the implementation of the project was affected by the insecurity situation in Gnagna province, which had caused the closure of the initially pre-selected schools ( Our hearts go with the children, teachers and parents from these schools. We hope things work out the best for them) . For this reason, activities are reoriented towards new schools located in a safer area of the same province, the commune of Bogande, and the project has been extended until the end of June. Apart from this initial delay, the project has been progressing very well. To achieve the goals, WASH has managed to get students, local authorities and communities, local organizations and private sectors involved throughout the project. Following work have been carried out:

  • Improving access to sustainable water and sanitation facilities by constructing and rehabilitating water points and latrines, also via distribution of WASH kits.
  • Increasing knowledge on good hygiene practices by providing training for Hygiene Clubs, implementing Schools Action Plan and running awareness raising campaigns.
  • Strengthening schools and communities’ capacities by providing training to Parents’ Associations and Teachers.

Doesn’t this joyful smile make you feel happy too? :-).

Igalia CSR

Igalia helps building a library in Yoff

People in Yoff Senegal are expecting to have a library built at the ground floor of a local school named “Coruña” in 2024. Thanks to the “A library in Yoff” project.

The “A library in Yoff ” project is led by Ecodesarrollo Gaia, a NGO based in A Coruña, Spain. Ecodesarrollo Gaia is the founder for Yoff Coruña school. As part of their educational project, this effort aims to provide a safe and peaceful place for the local community to access books and other educational and cultural resources, and to create more jobs and professional development opportunities for the local community. Above all, Ecodesarrollo Gaia would like to get students from the school involved in this project.

Igalia has been working with Ecodesarrollo Gaia since 2018 and is very proud to provide full funds for this project. The funds cover building construction work, acquiring essential furniture, creating foundational bibliographic batches, providing computer equipment and digital media, and carrying new staff employment and training.

This 1 year project has been progressing well. At the time of writing, the constructor who built the school has scheduled a meeting in June to specify the details of the construction. People in charge of the municipal library Sagrada Familia of A Coruña have been contacted to prepare a training course in Yoff at the beginning of 2024. At the moment, some members of Ecodesarrollo have traveled to Yoff and stayed locally to help run the project.

A lot to look forward to!

, ,

Igalia CSR

Igalia on Reforestation

Igalia started the Reforestation project in 2019. As one of the company’s Corporate Social Responsibility (CSR) efforts, the Reforestation project focuses on conserving and expanding native, old growth forests to capture, and long-term storing, carbon emissions. Partnering with Galnus, we have been working on reforestation of a 10.993 hectares area with 11,000 trees, which absorbs 66 tons of carbon dioxide each year.

Phase I – Rois

The first land where the project started was framed in the woods of the communal land of “San Miguel de Costa” in Rois. Rois is a municipality of northwestern Spain in the province of A Coruña in the autonomous community of Galicia. Environment in this land was highly altered by human action and the predominance of eucalyptus plantations left few examples of native forest.

After the agreement for land transfer signed on the 29th June 2019, the project started with creating maps and technical plans for the areas affected. Purposes of these plans are not only to build a cartographic base for management and accomplishment of the works but also to designate accurately these zones as “Reserved Areas” in future management plans.

Work carried out

  • Clear and crush remaining eucalyptus and other exotic plants

Eucalyptus is an exotic and invasive species. It spread widely and had a negative impact on the environmental situation of this habitat. This work is to clear the existing growth and eliminate the regrowths of eucalyptus and other exotic plants before the start of planting work. In Q1 2020, 100% of exotic trees were cut as part of the Atlantic forest sponsorship. After that the focus was moved to eliminating the sprouts of eucalyptus grown from the seeds present in the soil in early years. Sprout and seeds elimination continues throughout the duration of the project.

  • Acquire trees, shrubs and all other materials needed.

To improve forest structure and increase biodiversity for the affected area, the following native trees and shrubs were chosen and purchased in Q3 2019 –
– 1000 downy birch (Betula pubescens)
– 650 common oak (Quercus robur)
– 350 wild cherry (Prunus avium)
– 150 common holly (Ilex aquifolium)

A lot of other materials such as tree protectors, stakes etc. were also acquired at the time time.

  • Planting

Planting started in Q4 2019, downy birches (Betula pubescens) and some common oaks (Quercus robur) were the first batch planted. This planting campaign continued in 2020 on the arrival of the rest of the trees.

  • Care during the first year

The first year after planting is key to the future development and success of the project.

In Q1 2020, 100% of planted trees were already sprouting and alive. At this stage 100% of the restoration area has already been planted with at least 65% of the trees.

The first summer after planting is vital for the trees and shrubs to settle. During the first summer, trees adapt and take root in their new location. If they survive, their chances of developing correctly will increase exponentially and the following years they will be able to focus all their energy on their growth. The beginning of the summer 2020 was difficult due to hot and dry weather. However, the tree mortality rate remains within the expected range.

In Q4 2020, most of the planted trees are well-established and ready for spring sprouting. By Q1 2021 many of them have reached over 2 meters high.

The trees and shrubs settled happily after the first year of growth. This is the photo taken in Spring 2020

And see how they had grown in a year’s time –

  • Wildlife studies

With plants sprouting all around the forest, insects, reptiles and birds start thriving too. The development of the ecosystem will provide for better results in future inventories and wildlife studies. In summer 2020 the wildlife studies started and first inventory was completed. In winter 2020 wildlife nest boxes were in the manufacturing process and some bat boxes were installed in Spring 2021.

  • Improving biodiversity

In addition to ensuring the introduction of a wide variety of tree species to reforestation areas, the project has also put effort on enhancing biodiversity in existing forest areas. For example, One of the areas targeted by the project is one hectare of young secondary forest made up mainly of oaks (Quercus robur). In this forest we have been planting understory species such as holly (Ilex aquifolium), laurel (Laurus nobilis) or Atlantic pear (Pyrus cordata) to increase biodiversity and improve the structure and complexity of the forest.

Phase II – Eume and Rois

Phase II of the reforestation project set sail in winter 2020. While the Rois project had moved to a steady stage with most trees and shrubs planted and settled through the first year’s growth, this phase is to expand restoration of new forest area in Rois and to start a new project in “Fragas do Eume” natural park.

  • Rois

In order to explain the progress in the Rois expansion project, the maps below distinguish three areas in different stages of development.
– Phase 1 (Green) – The green area is completely planted with native trees and free of eucalyptus sprouts.
– Phase 2 (Yellow) – Acacias have been eliminated in this area. The work to control this species and also the eucalyptus trees will continue.
– Phase 3 (Red) – The entire area is covered with dense plantations of eucalyptus and acacia. Work is in the preparation stage.

The following maps represent the progresses made between Spring to Fall in 2021.

Spring 2021:

Fall 2021:

  • Eume

Unlike the Rois project, habitats in “Fragas do Eume” natural park are some of the largest and best preserved examples in the world of coastal Atlantic rainforest, where much of its biodiversity and original structure is still preserved. Unfortunately, most of these forests are young secondary forests under numerous threats such as the presence of eucalyptus and other invasive species.

Our project represents one of the largest actions in recent years for the elimination and control of exotic species and the restoration of native forest in the area, increasing native forest surface area, reconnecting fragmented forest patches and improving the landscape in the “Fragas do Eume Natural Park”.

In order to explain the progress in the Eume project, it also distinguishes three areas in different stages of development.

Phase 1 (Green) – All the environmental restoration works have been completed. Only maintenance tasks remain for the next few years.
phase 2 (Yellow) – This area is in the process of control and elimination of eucalyptus to start the restoration of native vegetation with native trees plantation works.
Phase 3 (Red) – Waiting for the loggers to cut the big eucalyptus trees in this area, in order to start the work associated with the project.

The following maps represent the progresses made between Spring to Fall in 2021.

Spring 2021:

Fall 2021:

Esmelle

In 2021 Igalia started working on a 0.538-hectare land owned by the Esmelle Valley Association. This land is borded by a road and a forest track on each side, and the Esmelle stream flows across it. This once mighty stream and its tributary springs are currently used to provide drinking water to the houses in the surrounding area. Unfortunately, in summer time, the flow of the stream could reduce drastically due to dry weather and exhausting uses. In addition, this area also suffers invasion of numerous eucalyptus trees and other exotic species. Recovering and restoring a good example of the Atlantic Forest will bring great benefit to this enormously altered and humanized case.

After work preparation and planning, major work were carried out in 2022 including –
– Elimination and control of eucalyptus and its sprouts.
– Clearing and ground preparation
– Tree seedling, plantation and protections (placement of tree stakes and protectors etc.)
– Maintenance and replacement of dead trees

This work will continue in the next two years. Main focus will be on reinforcement and enrichment of the plantation based on the availability of tree seedlings of certain species, and finishing all the remaining maintenance work.

What’s next?

Igalia sees Reforestation as a long term effort. While maintaining and developing the current projects, Igalia doesn’t stop looking for new candidates. In Q4 2022, Igalia started preparing a new Reforestation project – Galnus: O Courel.

This is the first time that Igalia considered developing an environmental compensation project in one of the most rural and least densely populated areas of Galicia. Igalia believes that the environmental, social and economic characteristics associated with this area, offers an opportunity to carry out
environmental restoration projects. If it goes as planned, major work will happen in the next two years. Something to look forward to!

Impacts

The Reforestation project is making contributions to our environment, and it has gone further. Here we’d like to share a picture of a 7-year-old boy’s work. This young student is from the community owning the Rois forest. He took advantage of a school newspaper project to communicate about our work in Rois forest.

Isn’t it a joy to see Iglaia’s Reforestation project is making impacts on our children, and our future? 🙂

Igalia Web Platform Team

2022 Review – Igalia Web Platform Team

2022 was a good year for the Web Platform team in Igalia in a lot of ways. We worked on some really interesting Web technologies, and explored some really interesting new ways of funding them. We hired some great new people, and attended and spoke at some great events. And we did some of our meetings in person!

Projects

Open Prioritization Projects

In 2020 Igalia launched the Open Prioritization experiment. It is an experiment in crowd-funding prioritization of new feature implementations for web browsers.

  • :focus-visible in WebKit

Our first Open-Priotization experiment to choose a project to crowdfund launched with 6 possible candidates representing work in every browser engine, across many areas of the platform and at various stages of completeness. According to the pledges that different projects got, :focus-visible in WebKit stood out and became the winner. Rego worked on this feature in 2021 and he had been reporting regularly about his progress. :focus-visible was finally enabled by default in WebKit and shipped in Safari 15.4.

Bear in mind that Apple Did Not Crowdfund :focus-visible in Safari. Eric made it very clear in his blog that, “the addition of :focus-visible to WebKit was lead by the community, done by Igalia, and contributed to WebKit without any involvement from Apple except in the sense of their reviewing patches and accepting the contributions”.

  • Enter: Wolvic

In February 2022, Igalia announced Wolvic, a new browser project with an initial focus of picking up where Firefox Reality leaves off. Brian published Enter: Wolvic explaining what WebXR could bring and Igalia’s initial focus with Wolvic. As an open collective project, Wolvic Collective is seeking more partners and collective funding to further explore the potential and capability of WebXR technology. To understand in details about this project, please follow Brian’s talk on Wolvic’s vision.

The igalians involved in the Wolvic project have put a lot of effort on updating and nurturing the Firefox Reality codebase. Follow the commits by Sergio (@svillar), Felipe(@felipeerias) and Edu (@elima) from Web Platform team at Wolvic Github codebase.

  • MathML

As part of the Open Prioritization effort, it’s exciting that MathML is going to be shipped in in Chrome 109. MathML collective is also part of our open collective effort. The plan is to have MathML-Core support in other browsers for browser interoperability. The MathML-Core support is level 1 for this project. We have envisioned more work to follow and will move to Level 2 after that. Your supports would make a lot of difference to move this forward.

Other Collaborative Projects

Most projects happening in the team are collaborative efforts with our clients and other parties in the Open Source community.

  • :has()

The :has() pseudo class is a selector that specifies an element which has at least one element that matches the relative selector passed as an argument. Byungwoo (@byungwoo_bw on Twitter) started the prototyping work since 2021 and It’s shipped in Chromium 105 this year.

CSS :has() pseudo-class is a very exciting feature. Igalians have been talking about it since its initial stage of filing an intent to prototype. Byungwoo presented implementation and performance details in Blinkon 16. And you can find a recent blog on examples of using it by Eric. In addition, Byungwoo had a talk on Updates since has was enabled in Chrome.

  • CSS highlight pseudos

Since 2021, Delan has been working on spelling and grammar features in Chromium. She has been writing a series on development of this work ( Chromium spelling and grammar features and Chromium spelling and grammar, part 2). As the project progresses, Delan produced her third part of the series on CSS highlight pseudos.

  • Custom Handlers component for Chrome

Igalia has been working on improving the integration of the IPFS protocol in the main browsers. Javier has been blogging on his work including New Custom Handlers component for Chrome to introduce some architectural changes in Chrome to move the Custom Handlers logic into a new component, and Discovering Chrome’s pre-defined
Custom Handlers
to discuss feasible solutions.

  • Automated accessibility testing

The Web Platform team has been engaging in various aspects of accessiblity work in Browser, from web standard to solutions. The Automated accessibility testing by Alex is a reflection of our knowledge on testing.

  • WPT Python 3 Migration

In 2020, Igalia was involved in the Python 3 migration work for the web-platform-tests (WPT) project. The WPT Python 3 Migration Post walks through challenges, approaches and some implementations for this migration work.

As always, there are a lot of other Web technology topics Igalians are working on. We recommend you to follow Brian’s blog and Eric’s blog for updates.

Conferences & Events

The Web Platform team were able to attend some conferences and Events on-site this year after the Covid-19 Pandemic. Here are some highlights.

  • Igalia Summit and Web Engines Hackfest

With employees distributed in over 20 countries globally and most are working remotely, Igalia holds summits every year to give the employees opportunities to meet face-to-face. The summit normally runs in a period of a week with Hackfest, code camps, team building and recreational activities etc..

As it is a great opportunity to bring together people working on the different browsers and related standards to discuss ideas and plans, during the summit week Igalia normally hosts Web Engines Hackfest to invite members from all parts of the Web Platform community. This year’s Hackfest was a success with over 70 people attendees during 2 days involving lots of discussions and conversations around different features. Rego noted down his personal highlights for the event. Frédéric also published the status Update on OpenType MATH fonts.

The summit is a great event for Igalians gathering together. For a new joiner, it’s a great first summit. And for an experienced Igalian, there are a lot to take away too.

  • Blinkon 17

Blinkon 17 in November was a hybrid event hosted in the San Francisco Bay area as well as via the Internet. Igalia had a strong presence in the event with 7 Lightning Talks and three breakout talks. From the Web Platform team, Delan presented her work on Faster Style & Paint for CSS Highlights and Frédéric updated the status on MathML in his talk Shipping MathML in Chrome 109 in breakout sessions. Andreu’s lightning talk on Specifying Line-clamp is very interesting. And you might find the live session on History of the Web by Brian and Eric fascinating to follow.

  • TPAC

TPAC this year is a hybrid event and rego was happy that he managed to travel away to TPAC 2022 after pandemic and meet so many people in real life again.

Hiring

Igalia has been hiring actively . Web Platform team is no exception. In his blog of Igalia Web Platform team is hiring , Rego explains how Igalia’s cooperative flat structure works and what you could expect working in Igalia. Web Platform team is a very friendly and hard-working team. The team’s blogs probably only show a small part of work happening in the team. If you’d like to know more, contact us. And even better if you are interested in joining the team, check on rego’s blog about how to apply.

We were all super excited that Alice Boxhall (@sundress on Twitter) joined the team in November 2022. Alice represented Igalia as a candidate for the W3C TAG election. Brian tells you in his blog Alice 2023 why we thought that Alice could be a good addition to the W3C TAG. Although the final result didn’t go our way, our support for Alice continues for the future :-).

Looking forward to 2023

With some exciting work planned for 2023, we are looking forward to another productive year ahead!

, ,

Igalia summit

My first time attending Igalia Summit

With employees distributed in over 20 countries globally and most are working remotely, Igalia holds summits every year to give the employees opportunities to meet up face-to-face. The summit normally runs in a period of a week with code camps, team building and recreational activities etc.. This year’s summer summit was held between the 15th-19th of June in A Coruña, Galicia.

I joined Igalia in the beginning of 2020. Because of COVID-19 pandemic, this was my first time attending an Igalia Summit. Due to personal time schedule I only managed to stay at A Coruña for three nights. The overall experience was great and I thoroughly enjoyed it!

Getting to know A Coruña

Igalia’s HQ is based in A Coruña in Galicia, Spain. A beautiful white-sanded beach is just a few yards away from the hotel we stayed. I did a barefoot stroll along the beach in one morning. The beach itself was reasonably quiet. There were a few people swimming in the shallow part of the sea. Weather that morning was perfect with warm sunshine and occasional cool gentle breezes.

The smell of sea in the air and people wandering around relaxingly in the evenings, somehow brought a familiar feeling to me and reminded of my home town.

On Wednesday I joined guided visit to A Coruña set in 1906. It was a very interesting walk around the historic part of the city. The tour guide Suso Martín presented an amazing one man show by walking us through the A Coruña‘s history back to the 12th century, historic buildings, religion and romances associated with the city.

On Thursday evening we went for the guided tour of MEGA, Mundo Estrella Galicia. En route we passed the office that Igalia used to locate at in early days. According to some Igalians who worked there before, it was a small office and there were about just over 10 Igalians in those days. Today Igalia has over 100 employees and last year we celebrated our 20th year anniversary in Open Source.

The highlights of the MEGA tour for me were watching the product line working and tasting the beers. We were spoiled by beers and local cheeses.

Meeting other igalians

Since I joined Igalian, all the meetings happened online due to pandemic. I was glad that I could finally meet other Igalians “physically” . During the summit I had chances to chat other Igalians at code camp, during meals and guided tours. It was pleasant. Playing board games together after an evening meal (I’d call it “night meal”) was a great fun. Some Igalians can be very funny and witty. I had quite a few “tearful laughter” moments. It was very enjoyable.

During the team code camp, I had chances to spending more time with my teammates. The technical meetings in both days were very engaging and effective. I was very happy to see Javi Fernández in person. Last year I was involved in CSS grid compatibility work (one of the 5 key areas for the Compat2021 effort that Igalia was involved in). I personally had never touched CSS Grid layout at the beginning of the assignment. The web platform team in Igalia though has very good knowledge and working experiences in this area. My teammates Manuel Rego, Javi Fernández and Sergio Villar were among the pioneer developers in this field. To assure the success of the task, the team formed a safe environment by providing me constant guidance and supports. Specifically, I had Javi’s helps throughout the whole task. Javi has been an amazing technical mentor with great expertise and patience. We had numerous calls for technical discussions. The team also had a couple of debugging sessions with me for some very tricky bugs.

The team meal on Wednesday evening was very nice. Delicious food, great companions and nice fruity beers – what could go wrong? Well, we did walk back to the hotel in the rain…

Thanks

The event was very well organized and productive. I really appreciate the hard work and great effort those igalians put in to make it happen. Just want to say – a big THANK YOU! I’m glad that I managed the trip. Traveling from the UK is not straightforward but I’d say it’s well worthy it.

,

WPT

WPT Python 3 Migration

In 2020, Igalia was involved in the Python 3 migration work for the web-platform-tests (WPT) project with sponsorships from Google. After a year-long effort, in December 2020 the flag for python 3 was switched on in WPT. Now over a year on, I only just manage to write about this migration work.  Better late than never, I hope :).

Why migrate?

Python 2 came to the end of life (EOL) on the 1st of January 2020. It marks the end of bugfix support or even security patches for Python 2 from Python maintainers. Code for the final Python2 release  2.7.18 ( happened in April 2020) was also frozen in January 2020. As a well used cross-browser test suite for the Web-platform stack, the web-platform-tests (WPT) Project uses python in many places, from infrastructure to test scripts. From maintenance and support for active development points of view, It’s imperative for WPT to make its code PY3 compatible sooner than later.

Challenges

Both the dynamic quality of the Python language and the complexity of the WPT present significant challenges to the upgrade.

Language challenges

Python is a dynamically typed language. There are no formal semantics for Python. As its de facto reference implementation, CPython maintains high coding standards but is not written with legibility as its primary focus. This means that code paths in Python can contain illegal semantics that are hard to detect even with non-static analyzers. Python 3 is a new version of Python, but it’s not backwards compatible with code written for Python 2. The nature of the changes between Python 2 and Python 3 are not just syntactical, rather, many of the changes are in the semantics. In particular, string literals are fundamentally different types in Python 2 and Python 3. Along with the change in the nature of the language, library support has also shifted. Many older libraries created for Python 2 are not forward-compatible. A lot of recent developers are creating libraries that can only be used with Python 3. We can run tools such as  caniusepython3 to take in a set of dependencies and then figure out which of them are holding us up from porting to Python 3. The tricky part though, is find and port the new libraries that will work.

Project challenges

WPT is a massive suite of tests (over one million in total), and serves many auxiliary functions. It uses Python in many places including but not limited to:
  • The majority of the infrastructure code. This is the code underlying the major wpt command, such as ‘wpt runner’ etc..
  • WPT file handlers, which test authors can define to run custom code in response to them making a particular request to the WPT server.
  • WebDriver tests, which use pytest structured tests.
  • Linting
  • Interacting with the docker, CI systems
  • Rebasing expectations, …
The complexity of the code base requires us to take a step back and have a good overview of the relations of the components that are involved and make a good plan on porting principles, pathways and methodologies.

The Porting Plan

The WPT community was well aware of the challenges of moving to Python 3 for the project. It set principles, suggested possible approaches and planned timelines before and during the major practical work took place.

Principles

  • The migration work should happen in the background since the project is quite active. 
  • The pathway to Python 3 was to make code dual Python 2 and Python 3 compatible and gradually switch over the runtime to Python 3. 
  • The porting should not reduce test coverage without explicit agreement from test authors.

Approaches

To make the porting tractable, it was decided to start with two very specific goals, each approaching the problem from different angles. One was to get the actual runner utility up running in Python 3, by starting to get a basic ‘wpt run‘ command to execute under Python 3. The other was to target wider test coverage via tests by running all relevant unit tests under Python 3.

TimeLines

For a project of non-trivial size like WPT, flag day transitions from Python 2 to Python 3 were simply not viable at the early stage of the project. Before 2020, there were already a few in-depth discussions and work going on within the community for the migration work. The major work, though, happened in 2020.  As the porting progressed, the timelines had got clearer. A concrete timeline of dropping Python 2 support in WPT was set in September 2020:
  • Py3-first” targeting 2021-01-01 : switch test runs to Python 3 on CI, but keep running unit tests and infrastructure tests in Python 2 and 3.
  • “Py3-only” on 2021-02-01: drop all Python 2 tests from CI, and start accepting Python 3-only changes.
WPT successfully moved to the “Py3-first” stage before the targeted date. The minimum python 3 version supported for this move is 3.6 with main focus on 3.8+. 

Implementations

Porting test runner utility

As we mentioned earlier, one of the starting points was to have the actual runner utility, ‘wpt run’  command to execute under Python 3. This porting was pretty straightforward. We came across some typical python 2 to python 3 migration issues such as
  • absolute imports. Absolute imports have become the default in Python 3 and relative imports should be explicit. For example, “from conftest import product, flatten” in Python 2 needs to be declared as “from .conftest import product, flatten” in Python 3.
  • built-in types comparison. In Python 3 most objects of built-in types compare unequal unless they are the same object. The choice of whether one object is smaller or larger than another one is made arbitrarily but consistently within one execution of a program. In Python 2 in the case of ‘mismatched’ types, the types are listed lexicographical by type name, e.g. a “list” comes after an “int” in alphabetical ordering, so is greater. For example, in Python 2, we have

latest_release = 0
version = [int(item) for item in m.groups()]if version > latest_release:

This is not valid in Python 3. Rather, we need to declare latest_release as latest_release = (0,0,0)
  • API changes. There are some API changes between the two versions. For example, the changes of the optional parameter strict in HTTPConnection(). In Python 2 we have httplib.HTTPConnection(self.host, self.port, strict=True, **conn_kwargs). In Python 3 it has become HTTPConnection(self.host, self.port, **conn_kwargs)
  • order of dict. In Python 2, dict is organized via a hash-table and puts the keys into buckets according to their hash() value. in Python 3.6+, dict retains insertion order. One solution to make code work for both versions is to use the  alternative type OrderedDict instead of the original Dict in Python 3.
  • iteration. Python 3 changes the return values of several basic functions from list to iterator. The main reason for this change is that using iterators usually causes better memory consumption than lists. This change has little impact on common use cases. Furthermore, the iter* counterparts (which return iterators in Python 2) have been removed. To make code work for both version, we can call six library APIs and replace them with six.iter* to avoid memory regression in Python 2. This corresponds to dictionary.iteritems() in Python 2 and dictionary.items() in Python 3. six is a Python 2 and 3 compatibility library. It provides utility functions for smoothing over the differences between the Python versions with the goal of writing Python code that is compatible on both Python versions. We called the six  library APIs at a few places during the dual Python 2/3 compatible stage. These API calls were removed after WPT transferred to python 3 only.
  • Bytes vs. str. In python2, binary is basically an alias of str. In python3 the binary data is different to a string. We had to convert some binary data to string type in order to be compatible for both Python 2 and Python 3. This issue, at the utility script level, presented different challenges from that in the core level we are discussing in the next section. Most cases in the utility script can be resolved by adding prefix to quoted string literals. Quoted string literals can be prefixed with “b” or “u” to get bytes or Unicode, respectively. In another word, prefix a native string with “u” in Python 2 to get a Unicode object while prefix with “b” in Python 3 to get bytes. It is also noted that in Python 3, the “u” prefix does nothing. Likewise, the “b” prefix does nothing in Python 2. In the context of this blog, we are talking about prefixing a native string with “b” to get bytes in Python 3 in most cases. 
There were also a few other issues such as Integer division, use of exceptions and call of print but they were generally very minor and easy to resolve.

Handling string types in core

One of the biggest hurdles in our porting effort was how to overcome the string literals type mismatch between Python 2 and 3 in core, specifically in infrastructure and file handlers. As we discussed earlier, in Python 2, a string literal is a sequence of bytes. In Python 3, a string literal is a sequence of Unicode code points. The rationale behind the change was to move to a Unicode-by-default world. Web Platform Test Server (wptserve) often intends to use byte sequences. To overcome this mismatch hurdle, we need  to either always use byte sequences or always use str[RFC49] has illustrated pros and cons for both approaches. It was decided within the community to go the byte sequence path in order to keep a consistent and semantically correct encoding model. That is to always use byte sequences: str in Python 2 and bytes in Python 3. This had incurred some noticeable changes in WPT core. In wptserve
  • It introduced a pair of ISO-8859-1 encode and decode helper functions. Both of them can accept either binary or text strings, but always return binary/text strings respectively regardless of the Python version.
  • Most public APIs for custom handlers can only accept and return binary with notable exception of the response body.
In python file handlers, it has specified string types for Headers on both requests and responses, Request URL/form parameters and response bodies etc.. After the necessary changes in the core part were done, Robert Ma (@robertma) and Stephen Mcgruer (@smcgruer) from Google created  the porting guidelines. Based on the guideline, we re-examined pretty much every line of the  test scripts in the existing handlers to add prefixes to string literals when necessary. Here we’d like to walk through some examples on porting handler related tests following the guideline and hope to share some tips.

Writing Python 3 compatible tests

According to the guideline, rule of thumb for porting is to make sure all strings are either always text or always bytes; all string literals in handlers should be prefixed with "b" or "u".

Headers of request and response

Header data should always be binary strings for both keys and values. Prefer adding "b" prefixes to encoding/decoding.
  • The Request.headers dictionary-like interface (accessed via […], get, items).
headers = [(b"Content-Type", b"text/html")]
if b"allow_csp_from" in request.GET:
headers.append((b"Allow-CSP-From", request.GET[b"allow_csp_from"]))
  • The Request.headers.get_list method example:
assert isinstance(headers.get_list(b'x-bar')[0], bytes)
  • Response.headers.{get,set,append,update,items} examples:
response.headers.set(b'Access-Control-Allow-Origin', request.headers.get(b"origin"))
response.headers.append(b"Access-Control-Allow-Origin", b"*")

HTTP Basic Authentication

Request.auth.{username,password} are binary strings. For example, response.headers.set(b'Access-Control-Allow-Origin', request.headers.get(b"origin"))
response.headers.append(b"Access-Control-Allow-Origin", b"*")
response.headers.set(b'Content-type', b'text/plain')
content = b""

Cookies

  • Request.cookies (similar to Request.headers; it’s a MultiDict with all APIs of dict plus first, last, get_list). For example,
response.content = request.cookies[b"foo"].value
  • Response.{set,unset,delete}_cookie.
response.set_cookie(b"name", b"value")
response.unset_cookie(b"name")

Request URL/form parameters

  • Both the keys and values of URL/form parameters for the request (accessible via request.GET or request.POST) are all binary strings. Prefer adding “b” prefixes to encoding/decoding.
b"realm" in request.POST
request.GET.first(b"type", None) == b"value"

Response Status Message

  • Response status message is binary string as follows.
response.status = 401
response.headers.set(b'Status', b'401 Authorization required')
response.headers.set(b'WWW-Authenticate', b'Basic realm="test"')

Response body

The data put into the response body can be either text or binary strings, but the two types should never be mixed and string literals must be prefixed. response.writer.write(b"This is a body!")
return u”Hello, 世界!”

Status

WPT successfully moved to the “Py3-first” stage in December 2020. In February 2021 it dropped all Python 2 tests from CI, and started accepting Python 3-only changes.

, ,