Geena and Liz from the Karten Network took the Nuvoic Project on the road to the Communication Matters conference in Leeds in early September. They had lots of visitors asking about the project, and the photos and video clips of our participants really helped to show off everybody’s fantastic work and encourage new participants to join, so thanks to everyone involved for giving them so much to talk about!
One of our participants, Haroon Sahir, wrote the following piece about his experience of the project, testing the Voiceitt app and donating speech recordings through Voiceitt Ensemble:
It was just about the end of Lockdown when my main carer Leanne came back from her Covid-19 break. Before this I was just having three hours care a week during the ‘dark days of London Lockdown.’ When Leanne returned we were having telephone conversations with staff from the St Joseph’s centre in Hendon, London. We were trying to figure out on how to install Microsoft Teams onto my decade-old adapted I.T. equipment because nearly everyone was doing Zoom calls on their computers, mobiles or iPads and I wasn’t able to do this during lockdowns!’ Staff asked, after they noticed I was having speech impairment problems over the telephone, if I would be interested in taking part in a speech project. I said ‘yes’ and this led to the introduction to Geena, a technologist on the project, via e-mail.
We carried on by using the trusted and my best form of communication, which is email! There were lots of email exchanges between Geena from the Karten Network, St Joseph’s and me before I first met Geena with support from Leanne, travelling by London public bus to the St Joseph centre. That was start of the Voiceitt project for me. We did Zoom calls using Leanne’s mobile, using iPads to do my voice recordings, and Leanne videoed me using my speech recordings on the iPad in the gym and at the centre. During Geena’s home visits we set up my computer with Zoom and a microphone and she interviewed me and videoed me doing the Voiceitt recordings on my computer. Geena is coming again to try to set up our old 37″ TV in my bedroom, so I don’t have to use the awkward old Toshiba television remote control!
The most valuable and interesting thing about working on the Voiceitt project I did some 1729 Voiceitt recordings for Voiceitt Ensemble. When recording on Ensemble I felt I was working on my fourth unofficial internet job and more importantly in my opinion I was doing speech therapy independently without the help from a human speech therapist. In the future Voiceitt could be used as a science fiction version to speech therapy?
The interesting or funny thing was when I was doing the Voiceitt recordings at several night times my 77 year old mother came from her bedroom into my bedroom to see why I was talking to myself! Eventually Voiceitt Ensemble said “It has run out of sentences to record in this dataset”! I was the first person to complete all the phrases. I missed doing the recordings and hearing my speech back through the speaker/microphone which was one of the gadgets provided by the Voiceitt project and ordered by Geena via the internet!
*After this was written, the team released new sentences and Haroon has been busy recording again! You can also check out Haroon’s sports writing on Facebook. If you want to support inclusive speech recognition by donating non-standard speech recordings with Voiceitt Ensemble, please email Liz at liz@karten-network.org.uk.
One of our participants, Trisha from Northern Ireland, has been trying out the Smart Home functions in Voiceitt to help with baking cupcakes! She used Voiceitt commands to turn the mixer on and off via a Smart Plug, and to set a timer in Alexa for when the cakes would be ready. Trisha was one of our first participants, joining us through the Cedar Foundation to test and give feedback on the Voiceitt app. She’s now also donating speech samples through Voiceitt Ensemble. Thanks Trisha for all your amazing work so far! Check out her video:
‘Baking with Voiceitt’ video text description
The video starts by showing a young woman, Trisha, standing in a kitchen cracking eggs into a jug. She tips the eggs, some oil and some cake mix into a mixing bowl. She then uses the Voiceitt speech recognition app on her phone to say ‘Alexa, mixer on’. The app turns the mixer on using a smart plug, and later she says ‘Alexa, mixer off’ to turn it off. We see Patricia measuring the cake mix into cupcake cases then pass the tray to her support worker who puts them in the oven to cook. Trisha sets a timer by saying ‘Alexa set a 10 minute timer’ to the Voiceitt app on her phone. When they’re out of the oven, she tests them with a cocktail stick and smiles because they’re ready. When they’re cool she adds icing and decorates them with coloured sweets. She makes letters on each cupcake then arranges them in front of her to spell ‘Voiceitt’, says ‘Voiceitt’ and smiles.
Since our last update we’ve continued recruiting and supporting people with atypical speech to test and give feedback on the Voiceitt speech recognition app, and to donate speech samples to help improve the underlying technology. The app is designed to support communication and Smart Home control for people who have atypical speech, with details available on our project website.
Voiceitt at The Maples smart bungalow
In February, the Karten Nuvoic Project team visited The Maples smart bungalow at Portland College near Mansfield, to help make voice controls for their new Smart Home installation accessible for people with atypical speech, through the Voiceitt app. The team at Portland College were already supporting learners to try out a whole range of new equipment via Alexa, including some who use voice output communication aids (see their article in January’s newsletter for details). Using Voiceitt’s speech recognition app, the Nuvoic team made these commands accessible for people who want to use their own voice but who find that Alexa doesn’t understand their speech.
We setup Voiceitt commands to control lights and blinds in every room, to set the heating and view cameras at the front and back doors, and even show what’s in the fridge! With Voiceitt, users can choose the words they want to use and train the app to recognise their unique pronunciation. The Portland team have setup routines in Alexa to carry out several actions with a single command, which we replicated with Voiceitt. For example, ‘Good morning’ will open the blinds and turn on the lights throughout the bungalow, while ‘Good night’ does the reverse. ‘Let’s chill out’ closes the blinds and sets the lights low in the chill out room, activates a bubble tube via a smart plug and plays relaxing music. We also setup commands to control the Robovac (robot vacuum cleaner), and basic commands for the TVs using voice control through Voiceitt.
Sarah Field is a Learning Support worker at Portland College, and a participant in the Nuvoic Project. Sarah trained Voiceitt to recognise how she says each of these commands and demonstrates them in the following video.
It was a great opportunity to gain experience of using Voiceitt with a fantastic range of Smart Home equipment, which we hope to share with other participants and organisations – please email liz@karten-network.org.uk if you would like more information. Huge thanks to Sarah and the team at Portland College for allowing us to visit and for all your help while we were there!
Ensemble
In the latest phase of our project, we’re asking people with atypical speech to record and donate phrases through the Ensemble website, to help Voiceitt improve the way their speech recognition technology works. Thanks to everyone who’s enrolled so far, through Caritas St Joseph’s, Cedar Foundation, Enable Ireland, The Grange, Homefield College, Leonard Cheshire, Leuchie House and several individual participants too. Between them, they’ve donated over 3,500 recordings so far – a brilliant start!
We’re keen to include as many non-standard voices as possible so please get in touch if you or people you support want to add theirs to the Ensemble! We can offer equipment on loan, technical support, gift cards of up to £100 for participants and funding for organisations supporting someone to take part.
If you or your organisation would like to be involved in this exciting development work, please see our web pages or contact our project co-ordinator: liz@karten-network.org.uk for more information.
Other Updates
In March one of our participants, Daniel Burgess, presented his work on the Nuvoic Project to students at Oakwood Academy in Manchester. Daniel volunteers with Leonard Cheshire as a befriender at their Hill House service in Sandbach, and delivers Zoom karaoke and quizzes to residents. As well as testing Voiceitt and donating voice samples through Voiceitt Ensemble, Daniel also supports one of the residents at Hill House to take part in the project – thanks Dan for your amazing contribution!
Our Project Technologist, Geena Vabulas, has also recently presented our work with Voiceitt to over 100 Assistive Technology professionals working in HE and FE at the Jisc ‘Focus on the Future’ event. This generated a lot of interest and several new enquiries from organisations wanting to get involved – fantastic job Geena!
Get in touch!
We would love to hear from you if you, or someone you know may be interested in taking part in testing the Voiceitt app and/or donating speech samples. Please email our project co-ordinator: liz@karten-network.org.uk, or you can find more information and get in touch via our project web pages.
One of our participants, Louise, supported by FitzRoy in Sussex, has been using Voiceitt to control her TV more independently ? Check out their article for details…
In February 2022, the Karten project team visited The Maples bungalow at Portland College near Mansfield to setup and test Voiceitt commands to control their fantastic new Smart Home equipment, recently installed with funding from the Ian Karten Charitable Trust. We were able to use Voiceitt to control lights and blinds in every room, to set the heating and view cameras at the front and back doors, and even to show what’s in the fridge! The Portland team have setup routines in Alexa to carry out several actions with a single command. For example, ‘Good morning’ will open the blinds and turn on the lights throughout the bungalow, while ‘Good night’ does the reverse. ‘Let’s chill out’ closes the blinds and sets the lights low in the chill out room, activates a bubble tube via a smart plug and plays some relaxing music. You can also control the Robovac (robot vacuum cleaner) and give basic commands for the TVs using voice control, and we were able to replicate all of these through Voiceitt.
Sarah Field is a Learning Support worker at Portland College, and a participant in the Nuvoic Project. Sarah trained Voiceitt to recognise how she says each of these commands, and in the following video you can see her testing them out.
Through the visit, we were able to get experience using Voiceitt with a fantastic range of Smart Home equipment, which we hope to share with other participants and organisations – please email liz@karten-network.org.uk for more information.
Huge thanks to Sarah and the team at Portland College, for inviting us to visit and for all your help while we were there!
May marked the start of our second year working on the Nuvoic Project, our collaboration with developer Voiceitt, which aims to improve access to voice recognition technologies for people who have dysarthric speech. Voiceitt’s app supports people who want to communicate or control Smart Home technologies using their own voice, but who are often misunderstood by unfamiliar people or mainstream speech recognition. The Karten Network is leading on user involvement and testing in the UK and Ireland.
Updates
In April we were delighted to welcome our newest Technologist, Geena Vabulas, to the Nuvoic Project Team. Geena joins us part-time, bringing fantastic experience from her role as Policy Manager for Assistive Technology at Policy Connect, as well as previous experience in qualitative research and provision of support and training in assistive technology. Welcome Geena!
We’ve continued to work with Karten Centres and other organisations to recruit participants, and now have 46 people enrolled! Since our last update we’re pleased to welcome new participants from Caritas St. Joseph’s, Enable Ireland, FitzRoy, Oakley College and Portland College, as well as three new individuals who’ve joined us directly. They join those already involved individually or through Beaumont College, Cedar Foundation, The Grange Centre, Homefield College, Leonard Cheshire, National Star, New Bridge Horizons, SeeAbility, St John’s College and Young Epilepsy.
Testing is going well. We ask participants to choose useful phrases and smart home commands, train the app to recognise their spoken prompts and practise using them, then tell us what they think, including any problems or ideas for improvement. The Alexa and other Smart Home controls are really popular, especially to play music or radio. All users can…Continue reading about the Nuvoic Project
Get in touch!
We would love to hear from you if you, your organisation or someone you know may be interested in taking part, or if you’d like more information. Please email our project co-ordinator: liz@karten-network.org.uk, or you can find more information and get in touch via our project web pages.
Now almost 18 months into the Nuvoic Project we’re starting to see some great outcomes, both in terms of participants using the Voiceitt app to do more things more independently, and also some new developments in the user interface and app performance, in response to participants’ feedback.
The project is led by developers Voiceitt and aims to improve access to voice recognition technologies for people who have dysarthric speech. The Karten Network team leads on user testing, supporting people with non-standard speech to try the app for support with communication or Smart Home control, and give feedback on what works well and what could be improved.
Updates
We now have over 60 participants enrolled in the project, and we’re delighted to welcome new participants from Leonard Cheshire, Leuchie House in Scotland and Linkage College who join those already involved from Beaumont College, Caritas St. Joseph’s, Cedar Foundation, Enable Ireland, FitzRoy, The Grange Centre, Homefield College, National Star, New Bridge Horizons, The Oaks College, Portland College, SeeAbility, St John’s College and Young Epilepsy.
As Covid-19 restrictions have eased, more participants have been using Voiceitt to support communication out-and-about: to socialise, order food and drink in a café or to use public transport, for example. (Our team has also been able to get out and visit participants in person, which has been great!) We also have…Continue reading about the Nuvoic Project
Get in touch!
We would love to hear from you if you, or someone you know may be interested in taking part in testing and/or voice sampling. Please email our project co-ordinator: liz@karten-network.org.uk, or you can find more information and get in touch via our project web pages.