Fujitsu Limited is a Japanese multinational information and communications technology equipment and services company, established in 1935 and headquartered in Tokyo. They are a world-leading digital transformation partner. Using a wide portfolio of trusted technology services, solutions and products, they work with customers to co-create solutions that help them on their journey to enterprise-wide digitalization.
Fujitsu Limited operates on the global market and it provides services to companies all around the world. As such, the company realizes that communication barriers caused by different languages and disabilities can have a significant impact on the life of many people. With the impact of COVID-19, which severely reduced the ability to interact with people in-person, this problem has become more severe, due to the high volume of remote meetings and interactions.
Introducing LiveTalk
To overcome this challenge, Fujitsu Limited has developed an application called LiveTalk, which enables everyone to communicate without barriers. LiveTalk is able to instantly turn anyone’s conversation into text in real-time, no matter where they come from. LiveTalk, in fact, supports 43 languages, including Japanese and Chinese. It was designed with a real time voice recognition feature and real time subtitles for both in person meetings, as well as remote meetings and remote classrooms. It has been built to facilitate communication between deaf and hard of hearing people and hearing people, but its capabilities extend to real time language translation as well.
The application is targeting a very broad audience: from very young children to adults; from people who are learning Japanese as a second language to people with disabilities. The breadth of this project has introduced a few challenges:
- Japanese is based on three writing systems: hiragana, katakana andkanji. To make Japanese easier to learn, especially for very young children or people who wants to learn it as second language, Japanese includes special characters called ruby, which indicate how to pronounce the text. This makes the operation to render Japanese text in the application quite complex, because words must be wrapped considering the space taken by the Ruby feature.
- Accessibility is a key requirement. The application must provide features like support to high-contrast mode, so that it can be proficiently used also by people with disabilities.
- Since the audience can have different levels of familiarity with technology, the installation and update experience must be as simple as possible.
Fujitsu Limited was able to address all these challenges by leveraging WinUI, MSIX and Xamarin as foundation to the application.
Building a first-class experience with WinUI
WinUI was the natural choice to build a first-class experience on Windows, which could tackle all the challenges that the development team needed to address.
WinUI provides a powerful and extensible UI system, which enables developers to tailor the user experience based on their needs. The flexibility of the XAML framework enabled Fujitsu Limited to customize the TextBlock control, by integrating their own custom algorithm to render Ruby characters.
The application is now able to recognize the speech of the user and convert it into text using the standard Japanese characters. WinUI will do all the heavy work to add the Ruby characters on top and take in account the different spacing.
Another WinUI feature the development team took advantage of is built-in accessibility support. All the controls included in WinUI provides first-in-class accessibility features, by recognizing and adapting to the accessibility options that are available in Windows 10. Thanks to this feature, Fujitsu Limited has been quickly able to make the application theme aware, so that it can properly adapt to users who are using a light, dark or high contrast theme in Windows 10.
In the end, due to the flexible nature of LiveTalk, WinUI was the perfect choice to provide a user experience that spans across all the different scenarios where the app is used: from a traditional PC controlled by mouse & keyboard to a touch-enabled device like a 2-in-1 or a tablet. Thanks to the built-in support for multiple input experiences, Fujitsu Limited was able to quickly introduce specific modes to better support mouse, keyboard and touch.
Deploy and update the application with confidence using MSIX
MSIX, the innovative deployment solution for desktop applications on Windows 10, has helped Fujitsu Limited to provide a seamless and simple solution for all their users: consumers, enterprises, schools, etc.
By packaging the application with MSIX, Fujitsu Limited has published LiveTalk on the Microsoft Store, which enables a one-click experience to acquire the application. Additionally, thanks to the built-in automatic updates feature, users won’t have to take any manual action to make sure they’re using the latest and greatest version of LiveTalk. Windows 10 will take care of updating it automatically every time the Fujitsu Limited development team publishes a new version.
Reaching all the users with Xamarin
What makes LiveTalk a very powerful solution is its flexibility: it can be used during a live conference; during a remote meeting; or even just during an informal in-person chat with a colleague. As such, Fujitsu Limited needed to go outside the desktop to reach users wherever they are. Xamarin was the natural choice to bring the Windows application also to other platforms. By sharing a similar UI framework and the same .NET ecosystem as WinUI, Fujitsu Limited was able to reuse most of the investments they made to bring the application also on Android and iOS, including support for Ruby characters.
Fujitsu said "Microsoft understood our intentions better than we could and provided appropriate and specific advice. As a result, we were able to outperform our expectations for Ruby, minimize the impact of increased Ruby processing time, and plan that we can provide to the market with confidence. Regarding WinUI, the current situation and future vision became clear, and we were able to align Microsoft's WinUI roadmap with our product roadmap. "
Conclusion
Fujitsu Limited has now planned to align the product roadmap of LiveTalk with the roadmaps of WinUI and .NET. This choice has enabled the development team to have a clear plan on the evolution of the app. It will help them to continue researching new technologies for their product and to be confident that they will be able to quickly integrate all the latest enhancements in the Windows ecosystem.
