Paying technical debt in our accessibility infrastructure

Translations: es - Tags: accessibility, gnome

This is somewhat of an extended abstract for the talk I want to give at GUADEC.

Curently, very few people work on GNOME's accessibility infrastructure, which is basically what the free desktop ecosystem uses regardless of GUI toolkit. After Oracle acquired Sun Microsystems in 2010, paid work on accessibility virtually disappeared. There is little volunteer work on it, and the accessibility stack has been accumulating technical debt since then.

What is legacy code? What sort of technical debt is there in our accessibility stack?

  • It has few or no tests.

  • It does not have a reproducible environment for building and testing.

  • It has not kept up with the rest of the system's evolution.

  • Few people know how it works.

It's a vicious circle, and I'd like to help break the loop.

Quick reminder: What does the accessibility infrastructure do?

An able-bodied person with good eyesight uses a desktop computer by looking at the screen, and interacting with things via the keyboard and mouse. GUI toolkits rely very heavily on this assumption, and hardware does, too — think of all the work expended in real-time flicker-free graphics, GPUs, frame-by-frame profilers, etc.

People who can't see well, or at all, or who can't use a regular keyboard in the way applications require them to (only got one hand? try pressing a modifier and a key at the opposite ends of the keyboard!), or who can't use a mouse effectively (shaky hands? arthritis pain? can't double-click? can't do fine motor control to left-click vs. right-click?), they need different technologies.

Or an adapter that translates the assumptions of "regular" applications into an interaction model they can use.

The accessibility stack for each platform, including GNOME, is that kind of adapter.

In subsequent blog posts I'll describe our accessibility infrastructure in more detail.

Times change

I've been re-familiarizing myself with the accessibility code. The last time I looked at it was in the early 2000s, when Sun contracted with Ximian to fix "easy" things like adding missing accessible roles to widgets, or associating labels with their target widgets. Back then everything assumed X11, and gross hacks were used to snoop events from GTK and forward them to the accessibility infrastructure. The accessibility code still used CORBA for inter-process communication!

Nowadays, things are different. When GNOME dropped CORBA, the accessibility code was ported in emergency mode to DBus. GTK3 and then GTK4 happened, and Wayland too. The accessibility infrastructure didn't quite keep up, and now we have a particularly noticeable missing link between the toolkit and the accessibility stack: GTK removed the event snooping code that the accessibility code used to forward all events to itself, and so for example not all features of Orca (the screen reader) fully work with GTK4 apps.

Also, in Wayland application windows don't know their absolute position in the screen. The compositor may place them anywhere or transform their textures in arbitrary ways. In addition, Wayland wants to move away from the insecure X11 model where any rogue application can set itself up as an accessibility technology (AT) and sniff all the events in all applications.

Both our toolkit infrastructure and the world's security needs have changed.

What I have been doing

I've been re-familiarizing myself with how the accessibility stack works, and I hope to detail it in subsequent blog posts.

For now, I've done the following:

  • Add continuous integration to at-spi2-core. Three of the basic modules in the accessibility infrastructure — at-spi2-core, at-spi2-atk, pyatspi2 — didn't have any CI configured for them. Atk has CI, Orca doesn't, but I haven't explored the Orca code yet.

  • I intend to merge at-spi2-core/atk/at-spi2-atk/pyatspi2 into a single repository, since they are very tightly coupled with each other and it will be easier to do end-to-end tests if they are in the same repository. Currently those tests are spread out between at-spi2-atk and pyatspi2.

  • Made the README in at-spi2-core friendlier; turned it into Markdown and updated it for the current state of things.

  • Did a bit of refactoring in at-spi2-core after setting up static analysis in its CI.

  • Did a bit of exploratory refactoring there, but found out that I have no easy way to test it. Hence the desire to make it possible to test all the accessibility code together.

  • Fixed a crash in GTK when one uses gnome-text-editor with the screen reader on. (merge request awaiting review)

  • Currently trying to debug this missing annotation in gnome-shell.

A little call for help

Is anyone familiar with testing things in Gitlab that need to be launched by dbus-broker? I think this may require either running systemd in the CI container (a cumbersome proposition), or using a virtual machine with systemd instead of a container. Anyway — if you care about Fedora or dbus-broker, please help.