In the international conversation surrounding technology, accessibility is not often a word we hear. Popular papers tend to focus on scaremongering tactics generating headlines designed to create a fear towards AI innovation. But ultimately, AI can and will be used for good. So why is accessibility so important?
Accessibility is important because we have a social responsibility to ensure that everyone can participate in a rapidly evolving digital world. Accessibility is inclusive and creates equality. It gives individuals more independence on and offline and helps us create innovative solutions and better design practices through more intuitive interfaces. Accessibility is empowering.
One company doing this right now is Signapse, who are focusing on creating accessible technology for the deaf community. I was fortunate enough to speak with their co-founder and CTO, Ben Saunders, to learn about the work they are doing and the benefits it will have for the deaf community.
You can read it all below.
Well, first of all, thank you so much for agreeing to do this Q&A with me today. Could you tell us a little bit about yourself and how you've arrived at this point in your life?
So my background is in accessible technology. Sign language has been a part of my whole life. My mum was a special needs teacher and she worked with a lot of non-verbal children, and often when she taught them, she used sign language. So I was always learning a little bit of it when I was growing up thanks to her.
Then separately I became interested in technology, particularly artificial intelligence, and did a masters in AI about 5 or 6 years ago. Then I was really lucky to find the combination of these two passions of mine when I did a PhD at the University of Surrey; My research was fully focused on sign language translation, where the goal was to generate a sign language video translated from an English sentence. We've got a lab here of about ten PhD students and my supervisor has been working in the space for over 20 years.
One of the things that I was focused on was photorealism. This means generating sign language videos that look exactly like a human. That became the spark that started Signapse because we showed some of these videos to the deaf population and they got really, really excited by them.
The main reason for this was because the state of the art before us was using graphical avatars. These can often look really good and nowadays you've got Unity which makes them look really nice. But, they're obviously not human and we often find that the deaf community are a little bit offended by this. They are often quite simplistic and cartoony. Whereas when we showed them our photo-realistic videos they were like; “Oh, that's a human. We're used to that. That's amazing.”
So, for me, that was the turning point where we saw that there's definitely something commercial here, something that we can actually do in a real-life application to help the deaf community. I’ve really enjoyed my time in the research space but I feel like the best way to really make products that are going to help in the world was turning it into a company, which is why we started Signapse.
How do you go about converting the research that you're doing into a fully-fledged product that's ready to be commercialised?
It's a bit of a challenge at times. Research is amazing, you can do some really good work and the nice thing about it is that if it only works 80% of the time, that's okay because it's just research. But when you're turning it into a product, when you're turning it into a commercial application, it has to work 100% of the time. And so yes, it was a bit of a challenge for us to do a lot of the engineering work, rather than the research, to actually turn it into a real application that works.
What we found was some of the more advanced stuff we were doing in research is probably not ready for products yet. So we used mainly the video generation and the more simple translation and turned those into products. Stuff that we were a lot more confident already worked and was able to operate in the current tech space. Over time once that research develops a little bit more, then we can start moving that into products as well.
We are aware that when we move Signapse into the product space, we've got to be really conscious of the accuracy and the quality of those videos. To make sure that they actually are useful by the community and they're serving their purpose.
How does your product’s video generation work?
The main work that we do for the video generation is using an AI technology called Generative Adversarial Networks (GANs), which have been around for quite a while. Some of the stuff that I did in my PhD was applying GANs to sign language.
The way that we do that is by generating a video of someone signing. We can give an input of a pose sequence where the pose tracks all the upper body limbs of a person. So it tracks the arms, the hands and importantly the face movements that can be then input to this AI model that turns that pose into a photorealistic video. It can then generate someone with exactly the same pose, but now as a real video. If you do that over a full sequence with a full 200 frames of that pose sequence, you can then generate quite a nice looking temporally consistent output sign language video.
We base those videos off the appearance of real people, but they're generating sequences that they didn't sign originally. We're really careful with using people's appearances that we know and trust and that the person also trust Signapse. So every actor that we use is a deaf actor and we get their consent that they're happy for us to use their appearance in set commercial applications.
Right. So you’re making sure there's no risk of any issue arising down the line with people's images being used against their will then.
We would never use someone's appearance who hasn’t given consent. It's a really, really important thing for us that we started out quite early doors within Signapse. Because these products are going to be used for the deaf community and are really serving that purpose, we wanted all of our actors to be deaf actors, who get paid for their time and get paid when their appearance is used in commercial settings. Often these actors are within Signapse as well, they understand what we're trying to do and really buy into our mission.
Why is the product you are developing so important for the deaf community and how will it pave the way for future technical developments in this field.
Signapse’s goal as a company is to do automatic translation from an English sentence into a sign language video. We aim to get to the point where we can give it any sentence, like with Google Translate, and the video quality would be good enough. We're not at that stage right now, so Signapse has started working in some constrained domains.
The main domain that we started has been transport. The main reason for that is it's technically feasible and a good place for us to start, but also it's solving a real need. A lot of deaf people struggle with transport hubs. The information that's conveyed in those places is either on the tannoy announcements in an audio form, which is obviously completely inaccessible to them. Or on a screen with lots of English text, that sometimes they find a bit confusing as it can be hard when their first language is sign language.
It's quite a well-known problem in the deaf community. When there's a group of people staring at a board or we're waiting for the same train. Something happens on the tannoy, everybody moves apart from that one deaf person because they didn't hear that information.
So for us that's a real problem that we wanted to solve. Our sign language transport announcements give the same information that comes across the tannoy whilst also in a sign language format. This is something that we found really helpful and also address a real need in the community at the moment.
We host monthly deaf user groups and we were pushed towards this problem by them. We were asking things like; “we've got this technology, where do we think it could be used, where do we think it can be most helpful?” And that was the main place that they recommended. On top of that, we wanted to make sure that all that technology was actually helpful for the deaf community.
With regards to future technical developments, once our technology has matured and got to the point where we can do the full translation in any English sentence, we can see this increasing accessibility massively. There's a lot of use cases for this kind of technology, but we’re focusing on putting it in places where currently there's no interpretation. Currently there's just not enough interpreters so making it in a scalable way and making all that information really accessible to the community is a future goal for Signapse.
At the moment where is the Signapse product being trialled in the UK?
Our product is live in a few train stations in the UK. You can see our sign language video announcements that are announcing all departing trains from those stations. We're soon to be rolling out a few more stations that we are really excited about and more recently, we've moved across to America.
We found that American Sign language is quite simple for us to move the technology into. And so American airports are something that we've been targeting quite a lot recently. Soon, it will be released there as well. We find that public places are really helpful at the moment and want to serve those problems, but also get our name out there and have people to see our products.
What sectors outside of transport do you see your product being applied to down the line?
Once we've got that full automatic translation I can see massive potential use cases for this. One area that we know is an issue for the deaf community is websites. Websites often contain a lot of English text and a lot of jargon that specific to that use case and those are often quite inaccessible for deaf people.
At our monthly deaf user groups we often hear that for those deaf people, if they're looking at let’s say an NHS website, they often use family or friends who can hear to understand what that content is. There's currently no sign language interpretations on most websites and that's something we really want to address. We want to provide those translations on the website in sign language. And once our technology matures and is able to work from any English sentence, websites will become a really big use case for us.
The second one we're excited about is the broadcast sector. For example, at the BBC, all of their programs have closed captions and some of their programs do have signed interpretations. However, it's usually quite a small amount of programs, often only the more important ones, and we'd love to get to a point where we could provide interpretations for those kind of programs where currently interpretation doesn't exist.
We're always very conscious that sign language translators are amazing, they do an amazing job and that's always going to be a much better translation. But there's just not enough interpreters. For the BBC, I think it's only about 5% of their programs are signed. So if we could drastically increase that by providing automatic translation where the interpreters don't currently work, that would be a really big user base for us.
Something that I got really excited about at the beginning of my PhD was the idea that one day; in addition to turning on closed captions on Netflix, you can also turn on closed signing. That would be an exciting use case for me.
That's a good vision for the future of the product and the technology. On that topic, then, where do you see Signapse ending up in five years’ time?
So within five years, what I'd really love is for us to have the automatic translation product out there for both British Sign Language and also American Sign Language. We are based in the UK right now so British Sign Language is going to be our first market. But we're quite quickly being drawn towards America just because it's a more accessible place and there's a bigger deaf community there. So those would be the two big markets.
I'd love within five years to get to the same point as companies like Synthesia, where we could do automatic text-to-video avatars, as they've got a really nice platform. I'd love to get to a place where Signapse can offer a very similar product, but using sign language as well. Once we've got that technology to a place where we're really confident in it, and we like the qualities of the outputs, we can see this use case is massive.
The vision that we have for Signapse is: All the world's information right now is in a written and audible format, why can it not be in a sign language format? And we'd love to make that dream come true in the next five years.
That’s a great mission statement to have moving forward. How do you go about scaling up Signapse to reach that capacity?
I think at the moment we are quite new, we are quite young and based mainly on the research side, so we do have a lot of scaling and maturing to do. I think something that's going to be really important for us is scaling out our research team and having a more professional element to that. Maybe a bit more video effects as opposed to the current research. That's the main place that I can see us really expanding.
On top of that, there’s also more of a product focus at the moment, often because we've got a lot of behind the scenes research to do, the product is sometimes less focused on and so over the next few years our attention is going to turn to that a lot more. But also we want to scale up our deaf impact sector, in terms of having people within the company who understand the applications that we're doing for the deaf community. So that we can get real feedback from them, and use that to lead the journey for Signapse and where we go to next. That's going to be an important place for us to scale up, having internal deaf data collectors, having internal deaf accessibility experts. That side of it is going to become really important for us to scale over time.
Because a lot of this technology came from my PhD and a few of us in the lab, I think scaling is about getting it out of a few people's heads and turning it into a much bigger team. So yeah, there's a few issues on the way to getting there, but we’ll take it one step at a time.
One way I love to round off these Q&A sessions is with this specific question which is: Can you recommend a piece of literature or research to the reader of this article for them to check out?
For a technical reader, some of the papers I published during my PhD would be really interesting. There's one that I published at CVPR in 2022 which was called ‘Signing at Scale’, that was talking about this video generation tool, but also how we convert from the English sentences into sign language through a sign language representation called Gloss.
From a non-technical point of view, we do a lot of blogs and information on our LinkedIn page, educating people on why some of these products are needed and what the impact is going to be for the deaf community. That's an area that I'd really recommend to go to as well.
A special thanks to Ben for taking the time to speak with us for this Q&A.