The third, and final, day of Microsoft’s Build 2017 was met with a lot less flare, but no less innovation. The keynote presentation on Day 3 featured a video, rather than speakers, and focused on innovative technology for businesses. There was also a continuation of the AI theme that had been going on throughout the conference.
The video presentation began with technology being implemented in skiing instruction by PSIA (Professional Ski Instructors of America). Their goal was to combine the use of sensors, which could be worn by skiers, and AI to enhance their ability to educate on proper skiing technique. The sensors shown in the demo video, appeared to be small watch-like device. Several sensors placed strategically on the limbs of professional skiers, allow data analysts to see what exactly (in numbers) the difference was between a novice skier and a pro. Then using this data to help with instruction. More than this, however, it could be possible for an AI to learn from this and provide feedback automatically.
Though, naturally, this is not developed with disabilities in mind, I could easily see this being applied to physical therapy. There continues to be a glaring problem with physical therapy: the therapist cannot monitor patients every moment. If individuals could wear sensors that were designed to work with AI intelligence, and were calibrated by therapists to what optimal positioning should be for exercises per individual needs, therapists could review the data and determine if patients were following exercise regimens properly.
Having a small device, like the one featured in the video, would be crucial to this design. Many people undergoing a physical therapy regimen are not able to wear a heavy piece of equipment. The addition of the AI would enable patients to improve on techniques even while away from the physical therapist’s office. It could also prove to be a powerful tool when teaching clients to use mobility aids such as canes, walkers, and push-wheelchairs. Proper technique is key to avoiding injury, and a therapist can’t be around every moment during the training process. A set-up of this nature could potentially aid in safer mobility training.
There was another potential medical use demoed using the HoloLens. The demo for the HoloLens was done by having Steven Guggenheimer wear a HoloLens device at the same time as the people demonstrating, looking at the same image. Those viewing the video saw the same image they saw, rendered in 2D on the video.
The first demo was with Vizua technology. They have been working on integrating mixed reality into the medical field. In the demo, Guggenheimer was shown a 3D image of a heart which had been recreated layer by layer using CT scans. Because nothing is being computed or stored on the HoloLens devices directly, but rather the applications are making use of Azure (Microsoft’s cloud computing service), it’s possible for multiple people to see and manipulate the image at the same time.
Doctors frequently find themselves working in teams. While in the video it was mentioned that the technology can work all over the world, it is common knowledge that doctors struggle with communicating with other members on their team—even within the same hospital. Using HoloLens and Azure, it’s possible for a team to discuss surgery plans prior to surgery, or even simply discuss a patient case, without needing to schedule time to physically be in the same room.
This also has the potential to fix another flaw in the medical field: doctor-patient communication. Unless a patient has a background in medicine, it can be difficult to understand scans, numbers, and other data. HoloLens could allow for a doctor to present, not a model case, but a real visual image of what is involved in a patient’s body. It’s possible to zoom in, take apart, highlight, and in other ways visually explain to patients what is going on. A communication leap that is certainly needed within the medical field.
Medicine, however, is not the only HoloLens potential real-life application. The second demonstration proved that the device could have interesting applications in building design. Aveva showed how they could use the product on a potential oil platform. Through the demo Guggenheimer could review the platform as a flat blueprint, a scaled 3D, and a full-scale reimaging of what the future building would look like. While wearing the HoloLens it allowed him to move his head and look around at the area of the would-be oil platform and see on a real scale, how it might look. It’s also possible to put a model person inside the platform to check for height problems (if workers can reach everything), among other safety concerns.
This has so many interesting applications for architecture design, but especially when trying to get people to consider how to make their buildings not only ADA compliant, but go beyond what the ADA requires. This set-up allows for someone to test whether a potential building design would work for various disabilities. It would be possible to put in models of several types of wheelchairs, look at hallway structures (can someone who is deaf see around the corners?), ensure signs are all uniformly placed so someone who is visually impaired knows where to look, double-check that everything is reachable from a lower height, and so much more.
Even more interestingly, is the ability for collaboration. Azure allows for everything to be saved in a way that, if desired, it could be possible for a company to consult with another one that specializes in accessibility in buildings. Rather than complex scheduling and flying personnel around, it would be possible to look at a design even if the co-collaborators were hundreds of miles apart. Azure also allows for an app to connect with Cognitive Services, so it would be possible to get the help of AI to look for ADA infractions in potential designs.
It is possible to go beyond just the visual, however. HoloLens also has audio abilities, and in the demo, it was proven that it was possible to move around and hear sounds in the 3D model. This has a lot of potential for acoustic checks, making sure that areas such as classrooms, for example, have acoustic layouts that enable a better listening environment for anyone who is hard of hearing.
Disability advocates know that planning for accessibility needs to be done from the ground up, and technology, like what we have seen demonstrated during build, is enabling that to be a reality. Azure will open up a whole new level of collaborative potential, and will give app developers the ability to use AI to optimize efficiency. The HoloLens is providing a new tool for imaging that can be used in not only medicine, but also building design. In the coming year, and beyond, we can look forward to seeing more of these types of applications and, hopefully, as a society, we will come to realize how much easier it is getting to make things accessible to everyone.
Like this article? Subscribe to get our bi-monthly newsletters about our 100% accessible content.