13 January 2025
Sign of the Times
FingerDance’s new AI tool takes a giant leap in information accessibility
FingerDance co-founder Mr Gong He (MSBA '23) demonstrating SiLViA – the Sign Language Virtual Assistant.
Deaf and hard-of-hearing commuters who need help with routes, directions, announcements, or other MRT-related enquiries now have support, thanks to SiLViA – the Sign Language Virtual Assistant.
Piloted at Chinatown MRT station, SiLViA is the co-creation of FingerDance, a deep tech social impact start-up, and SBS Transit.
The virtual assistant features FingerDance’s proprietary Sign Language AI translation and AI avatar technology, offering accessible sign language information services. Deaf and hard-of-hearing commuters can say or type their queries on the screen, and SiLViA will respond through both spoken and sign language, ensuring that all commuters receive essential information in real-time. More importantly, SiLViA can also translate audio announcements into sign language.
Using AI to promote inclusion for the Deaf and hard-of-hearing communities
FingerDance was co-founded by Mr Gong He (MSBA ’23), a passionate technologist inspired by the challenge of communicating with his Deaf friends.
Realising the limitations of text-based communication, Mr Gong He and his other co-founders Mr Zeng Zhen, Mr Xu Xuanqi, Mr Xu Xuanzhuo and Mr Sun Hao embarked on a journey to leverage their technological expertise to bridge this gap. “We started to think about whether we can use our tech background to do something different. That’s the beginning of the whole journey,” he recalled.
FingerDance’s groundbreaking technology is transforming their vision of seamless communication into reality. The generative AI technology uses a diverse collection of real-world sign language data to translate spoken language into sign language.
In order to ensure the accuracy of their Sign Language large language model (LLM), FingerDance is working with The Singapore Association for the Deaf (SADeaf) to integrate Singapore Sign Language (SgSL) into their products. This close collaboration with SADeaf and the Deaf and Hard-of-hearing community ensures that those who use SgSL are involved throughout the development of the product. Their valuable contributions and feedback are vital to authentically and accurately recording the sign language used by the local community.
FingerDance also offers accessible sign language information services for web-text, news, events, and conferences through its AI technology. Supported by the robust core AI model, it ensures continuous technical support and updates, facilitating accessibility in both the physical and digital worlds.
Utilising FingerDance’s Sign Language AI translation
and AI avatar technology, SiLVia offers accessible sign language information services to Deaf and hard-of-hearing commuters.
Scaling up with NUS Enterprise and strong industry collaboration
Bringing their vision into reality proved challenging, especially in achieving product-market fit within a B2B environment. Convincing stakeholders in government and industry sectors required extensive market research and a clear demonstration of their innovative solutions' potential impact.
The team's belief in their product eventually paid off, as they successfully navigated these challenges with significant support from the
NUS Enterprise ecosystem. This invaluable support system provided mentorship, resources, and networking opportunities crucial to their success. Programmes such as the
NUS Graduate Research Innovation Programme (GRIP) enabled FingerDance to refine their products and business model, with the start-up benefiting from the insights and collaboration of industry experts.
In July 2024, FingerDance was announced as one of seven start-ups that would receive funding from the new Technology for Sustainable Social Impact (TS2) accelerator programme established by NUS Enterprise and the Singapore Centre for Social Enterprise (raiSE), which committed US$3.5 million to growing tech-based social enterprise start-ups.
The start-up will also work out of the new BLOCK71 Social Impact Hub opened by NUS Enterprise, a dedicated space for purpose-driven ventures to further develop their tech-based solutions to solve pressing social issues.
FingerDance has received positive responses to their products during their start-up journey. Companies and end-users look forward to using their products in the future. This feedback not only validates the team’s efforts but fuels their commitment to continuous iteration. “Every time we get positive feedback during our workshops with different users, it gives us the confidence that our work will be useful for them,” shared Mr Gong He. “In the future, we plan to integrate more sign languages in different countries and regions, using our solutions to benefit Deaf people all over the world.”
Mr Gong He explains the FingerDance technology to an audience at the opening of the BLOCK71 Social Impact Hub
By collaborating with associations and organisations worldwide, they aim to provide a global solution for the Deaf and Hard-of-hearing communities.
The team envisions a future where inclusivity is the norm, and communication barriers are a thing of the past. Their partnership with NUS Enterprise remains pivotal in their growth strategy, facilitating access to new markets, potential investors, and a supportive entrepreneurial community.
FingerDance is a trailblazer for inclusive communication in Singapore, with the potential to revolutionise the way the Deaf and hard-of-hearing navigate their daily commute. With ongoing support from NUS Enterprise, they are poised to make significant strides as they expand and innovate.
This article first appeared in Issue #43 (Oct - Dec 2024) of SPARKS, a publication of NUS Enterprise.