And now we are welcoming Yola Ode Lola for the gaps we enter it and she will explain you what does it mean?
Hello, thank you.
Thank you very much.
Okay, so a question I had having worked in tech for a while now to be fair.
What resources would we need to create a website for predominantly blind?
It's not working?
Yeah.
Oh, it's meeting.
That should do.
Hello.
Yeah, that's definitely on.
You can hear the difference, can't you?
Right.
So if you're creating a website for predominantly blind audience.
So assuming everybody or 90% of people who visited your website, you need to be blind.
What resources would you need to make sure they had the experience you wanted them to have?
What would you need to test the website to ensure a consistent experience across browsers and screen readers?
And just for those who are unaware, if you are blind and you are using, well, not just the web, but any kind of digital device.
It's likely you would use an assistive technology called a screen reader.
There are a bunch of different kinds of assistive technologies and screen readers is just one category of assistive technology.
And some screen readers can even speak to other assistive technologies like Braille readers as well.
So what would we check for?
What should we check for?
In this talk, I want to explore the gaps in interoperability data on the web and why we have them.
My name is Lola Adelola.
I am the founder of Lola's labs.
I do web standards stuff.
I call myself a web standards technologist, but really I just stole that from a job advert that I saw somewhere.
I'm also the co-chair of the W3C technical architecture group and the co-chair of the W3C documentation community group.
I'm also leading a project called the accessibility compact data, which is what this talk is going to resolve to.
And hopefully you learn more about that.
But technology is my side hustle I like to say.
And really I'm an artist and I explore a bunch of different things through painting and photography.
And writing my first degrees actually in creative writing.
Nothing to do with computer science at all.
And I hate cheesecake and I think everybody should too.
You can debate me about that later.
So let's revisit this question.
What resources would we need to create a website for predominantly blind audience?
In the first instance we'd want to make sure that the web features we used can actually be accessed by a blind users.
This may seem familiar to some of you.
Probably most of you.
I don't know how many people in the remote web developers.
But if you look in the developer console of any HTML page, you see the Dom tree,
which is also the HTML representation of the page.
So here we have our doc type declaration, the HTML tag.
Inside it are head and body tags.
And inside the head tag we have our meta and title tags.
Then inside our body tag we have the main tag.
And inside that we have a H1 paragraph and button.
We also have something called the CSS object model,
which is a representation of the CSS rule for the given HTML.
The rules we set here are to have a font size of 16 pixels in the body.
And make all the paragraph text be blue with the margin of 10 pixels around the paragraph.
And this is represented in a similar tree format.
The same rules we've defined are now have a clear hierarchy.
The body and paragraph rules are on the same branch.
While the font size branches from the body and the color and margin
brought from the paragraph.
These two combined, so our Dom tree and our CSS objects model,
they give us our render tree.
And the render tree is really what dictates the layout.
This is the render tree for the HTML and CSS we just created.
And this is everything the user would need to see,
assuming that they were not blind.
The render tree is important because it's how the browser determines the layout for the webpage.
And what site are the users should see on the page and how they should see it.
Blind users who use magnification tools or software blindness that affect how they perceive colours
and contrast would also benefit from the render tree.
In fact, most WCAG requirements are met with HTML and CSS.
So in other words, this is how WCAG requirements would even be presented to the web user.
However, for web users who use screen readers, they do not need the render tree at all.
So if we're building our website for blind users, they wouldn't use the render tree.
They don't really need to know the layout or the thing what they need to know are the accessibility semantics,
which is communicated in the accessibility tree.
The important thing here is that the accessibility tree is a separate structure,
and extracts the roles, names, and texts of the don't tree, allowing screen readers to interact with the necessary semantics to communicate to users.
Screen readers also communicate these semantics to other devices, as I mentioned earlier, such as brow readers.
Every browser does this differently, but this is roughly what you can expect all browsers to include in the accessibility tree.
We don't have any information in this accessibility tree about styles, and we have limited information about what's in the head.
Mainly the role, name of the document, and for each element in the body we have the role, name, text of the element, and if that element is focusable.
These are the important semantics of screen reader in general.
And so the don't tree has all the HTML where the accessibility tree has the semantically meaning for information for screen readers.
The don't tree standardized in the don't standard, which is worked on in the web-hyper-text application technology work in group.
It's a mouthful, what rig, some of you may be familiar with that acronym, which also works on the HTML standard.
This makes it easier for there to be interoperability between browsers, as they can agree on what should be in the don't tree.
In the panel that we just had, we heard the panelists talk about the work that goes into making sure that there is a consistent web experience across browsers regardless of the browser you're using.
However, the same is not exactly true for the accessibility tree.
Who actually decides what should be in the accessibility tree? This isn't arbitrary.
There's a whole ecosystem of specifications and APIs that govern this.
However, it's not standardized because of that the browsers have flexibility on how they can implement their accessibility trees.
So we're going to go in a bit more detail and forgive me if this gets confusing.
It was confusing for me as I was delving into things.
There are series of specifications that specify how web features are mapped to platform APIs.
When we are dealing with the don't tree, we only really care about the web browser.
When you're dealing with the accessibility tree, yes, the accessibility tree is a semantic representation of what is shown in a browser,
but it connects to operating system, platform APIs.
As you can imagine, each operating system has their own platform API, which I'll talk about more detail soon.
What Apple says a link is going to be completely different to what Windows says a link is.
And that's because within their operating system, there are other elements that are have nothing to do with the browser that may use the same roles.
And so they get to define what those are and how those look.
So as I said, there are a series of specifications that specify how web features are mapped to platform accessibility APIs.
But the four most relevant to people who work on the web are the HTML core and SVG accessible API mappings and the accessible name and description computation.
The core accessibility API mapping is a document which describes how user agents, so not just browsers, but in this case a web user agent browser,
should expose the semantics of web content languages to accessibility APIs.
For example, for accessibility API 1, the way role link should be sent to the API as role system link.
And you can see that in the example for the core on the first line there.
The HTML accessibility API mapping does something completely different.
It defines how user agents map HTML elements specifically and attributes to platform APIs.
So in this example, the anchor element, with a non empty htresh bring, should have the way RRL link.
And because the anchor element would have the way RRL link, when it comes to mapping to accessibility API 1, it would then use the role system link to map to that operating systems accessibility API.
And the HTML document, the HTML accessibility API mapping document goes through all the HTML elements and does the same thing.
And then we also have our accessible name and description computation.
Rolls aren't the only important semantics that need to be defined or accessibility API.
We need to define accessible names and descriptions to and this is all helpful when creating the accessibility tree.
So the DOM tree only has to consider the browser.
However, the accessibility tree has to also consider the platform and operating system level APIs, which are mostly closed source and proprietary.
And don't use with each other in the same way browser specifications do.
So each platform has a different way of interpreting the data it receives and requires different instructions to properly interpret that data.
And so each platform API is an additional variable that needs to be considered when creating web experiences for blind people.
And so this is an example of the core AAM, oops sorry, the core AAM.
Each platform API accepts different key value pairs per way RRL.
For I accessible to, which is a Windows platform API, it will accepts the role the states the interface which are required.
Whereas UIA, which is also a Windows API, will accepts the control tab and control pattern required.
So we will take our way RRL link and send depending on the platform API, send different data.
And you can imagine how confusing that would get.
There's no consistency across platforms at all.
And so each screen reader uses one of these platform APIs, depending on the operating system.
So draws an NVIDIA, which are both screen readers, both operate on Windows and use I accessible to or Windows UIA automation.
While voice-overs, we're on macOS and uses the macOS accessibility protocol.
And Orca is available on Nome, and that uses the Linux accessibility toolkit.
And so we have a group of spec definitions, the core HTML accessibility AAM appings, and the accessible name and description computation, which says how the accessibility treat is constructed.
And then the accessibility treat passes those semantics to various platform APIs, which pass it to the assistive technologies in this case screen readers, which then pass it onto the users.
These steps can be grouped into how the semantics in the HTML given page move from browser to operating system to user.
That was a lot.
If we wanted to build a HTML page for cited users, how would we know the paragraph h1 and button tags would render as we expect them to?
We could use the browser compatibility data table. This is on MDN on most MDN pages. You have a compact table at the bottom, which tells you the support of the feature per browser.
We could use can I use, which shows similar data.
If you wanted to get straight to it, it doesn't have necessary reference information, but it does show you the browser support.
We could use baseline, and this is even more streamlined, more simplified, and it uses a red yellow green system to kind of show whether it's something is widely available or not.
Or maybe we don't even want to leave our IDE, and as we're typing, support data comes up to show that a thing is supported.
This is currently available for CSS properties in JetBrains, WebStorm, and Visual Studio Code.
The good thing about this, though, is that all those options is that they actually all use the same data source.
So BCD, which is browser compact data, actually is the data that you see in the IDE integration in baseline in can I use in the browser compatibility data.
It uses the same source, which means that there's unification of data, right?
All that needs to happen is one place needs to be updated for everywhere to be updated and for developers to have accurate information.
But the BCD collector does not include the accessibility tree in its support data information.
It's only considered as the DOM tree, and as we've seen the four blind users, that is not enough.
So how do we know our website works with assistive technologies?
There's Alice support.io, which is a website that includes data specifically for the way our raw mapping and our attributes.
However, this may be difficult for developers to read and understand, because you need to know which roles map to which elements.
And the data is incomplete in some places, as it relies on volunteers to contribute and update.
There's also PowerMapper, which is another good source, takes it a step further and gives support data for the HTML element across screen readers and browsers.
However, this data is closed source, and while they have a lot of data, it's difficult to know where that data is updated, where to file bugs, et cetera, et cetera.
There's also the RAT project, which initially was mainly for the APG platform, which is the area author and practices guide patterns.
However, they have expanded it to be HTML elements too, but work is still ongoing in this space.
And then there's web platform tests, and I don't know if any of you have tried to read web platform tests or get useful information from web platform tests, but if you are not a browser engineer, even if you are a browser engineer, unless you're working on something very, very specific, it can be difficult.
As my colleagues said earlier, there are like 2 million web platform tests where the hell do you even begin.
The presentation and segmentation, there are many options for us to visit, but they all do different things and present data in different ways or a closed source, or all these other things.
And the W3C maintains that there is one web, however, the prioritization of sighted users means that blind users often have to experience multiple or incomplete webs.
This infrastructure disparity tell us then.
As it stands, a lack of reliable and easily accessible, inter-up data encourages web developers to create websites that employ these kinds of dark patterns, even if accidentally.
For example, there's an assumption that if a web feature is currently exposed in the DOM tree, then it's also exposed in accessibility tree.
This leads developers to not fully testing their websites, causing at best a poor experience and at worst an exploitative web experience.
I'm going to skip through some slides here just so that I have enough time for questions at the end.
So that brings me to the thing that I'm working on, which I think would be of interest to folks in this room, which is the accessibility compact data.
This is a project I'm working on, and it has a few goals, right? So the first is to define machine readable data that represents accessibility support for web features.
To integrate this data into MDN and other developer resources, to support developers in building accessible experiences from the start, and most importantly, improving web experiences for users of assistive technologies.
ACD is not an audit in store, and it does not define a baseline.
We want to provide support data for the browser that takes the accessibility tree into consideration, and support data to highlight when screen readers support a particular element, or if there are bugs.
And similar to BCD, we want to integrate into the places where developers already are.
We don't want developers to have to learn a new website or go to a different place to find accessibility information.
We want it to just be integrated in the tools and resources that developers already use.
And so for users of ACD, the impact will probably be that users will be more catered to, especially those who use assistive technologies, right?
Web developers will know where they may need to fill holes.
Assistive tech and browser vendors will be more aware of bugs, and spec authors will be encouraged to write accessibility tests for their web features.
And so we do have support from a bunch of different people.
Last year, we had support from the early project, open web docs, Microsoft Patrick, who was on the panel.
A moment ago, he's a big support of the work we're doing.
Mozilla, technological, Apple have helped us in some cases, and then we also have a bunch of indie accessibility professionals and experts who know how all of this works and who can dedicate their resources, their time and their knowledge.
And in terms of where we are with the project, we have now a kind of very, very rough demo of a collector, which goes into a platform test, gets the relevant test from a platform test, goes into a screen reader data source that we've identified, combines the two and says that, okay, this web feature is supported in this browser version and this screen reader version.
And we still need to do a lot of work there, obviously, but that's where we are with it.
And that's it for me. You can find me on blue sky or Macedon, Lola, Lola, Lola, you can follow the project on GitHub, Lola, I think it's actually now Lola's Dash Lab, Dash Accessibility Compact Data.
And if you'd like to sponsor us, you can sponsor this work at give.lolaslab.co.
Thank you.
So we have time for a couple of questions.
Great rock, thanks.
This might not be very relevant to the developers, but would it be easier if the screen readers talk directly to the browser instead of going through the platform APIs and all that mess?
As somebody who does not work on operating systems, you have to take my answer with a huge grain of salt.
I'm going to say no, mainly because the screen reader is an operating system level technology.
It's not a browser technology, and so the screen reader still needs to speak to the operating system anyway.
Like if we didn't have a browser, like actual browsers didn't exist, screen readers could still exist and still be very useful to blind people.
If somebody opens up a word processor application on their desktop on their device, the screen reader still needs to engage with that.
It still needs to know what a hyperlink is in the context of the processor, so the document and etc.
So I don't know that it would be easier.
Thank you. Any other question?
So with after collecting this data with their being interested in an initiative like baseline to try to bring it or just incorporate some of these things into baseline.
Okay, good. So we are talking with the people who are doing baseline to see what we can do to integrate this data into baseline.
It's a bit tricky because on one hand something shouldn't be baseline if it's not accessible.
The other hand we don't want developers to rely on baseline as a way to understand whether or not something is accessible, right?
Like baseline is a simplification of a simplification and really for especially for accessibility data you want to kind of understand the nuances, so you can know what gaps you may need to plug.
So we are still in discussion. We're going to be running a survey this year to really understand how developers view baseline and accessibility.
But yeah, it's a conversation that's happening.
Thank you. One last question.
Hi. So maybe something we notice is like as a small company it can be hard to focus on accessibility because it's complex, it takes a lot of work and you don't have a lot of users that maybe need accessibility features but it could help a lot.
Do you think like for example AI could help in this case with like is accessibility something that is good to understand for an AI and could help on the developer side would basically
Assisting in how do you make this website more accessible but maybe also from the browser side because you have some new modern browsers that basically take away.
The browser pages again and give more of like a human interfacing like chat or other kind of thing do you think like these things will will help.
I think it's still to be seen. I don't know if anyone has tried to even create a HTML button with AI but it's very it can't do that in an accessible way and you know to make a HTML button accessible is not difficult at all.
I think we've had discussions with a few different people who are interested in possibly integrating this data into AI tools so that at least the AI if a developer is using AI to build something at least the data that they're pulling from is as accurate as possible.
But I think it's a bit more complicated it really depends on the AI tool and where their data is coming from.
A lot of the data on the web at the moment is insufficient because a lot of websites are inaccessible so it's like you know you kind of just running around and circles at that point.
If we could integrate this into AI I think that's an interesting prospect and we have spoken to a couple of people about that.
And you are a very quick last question.
We are a quick and another perspective on this being able to know if a feature is a portal not is very nice and it's very important but as we as a developer sometimes to like to try and test on the real thing for example spin up a different browser to test if the rendering is correct on Firefox and will come whatever.
Would it be possible to use this data to later create some kind of simulation environment where one could for example spin up the screen reader and the stack or simulation of the stack that's going through and see exactly where the quirks are experiencing the first time.
Would that be possible to like get a better understanding of the actual user experience.
It's just a compatibility data.
I think that's actually possible today but don't hold me to that.
I don't know that you would need this data to do that basically.
I think you can do that now but also maybe I'm wrong about that.
I have a feeling my quote that's what some of my colleagues have done in other accessibility tools that they are building.
Cool.
Thank you very much.
Thank you so much.
Thank you.