Google open sources spatial audio SDK for realistic VR and AR experiences

Google is looking to improve virtual reality experiences with new experiments and SDKs. The company announced that it is open sourcing Resonance Audio, its spatial audio SDK released last year. Google is also providing new insights into its experiments with light fields, which are a set of algorithms for advanced capture, stitching, and rendering.

“To accelerate adoption of immersive audio technology and strengthen the developer community around it, we’re opening Resonance Audio to a community-driven development model. By creating an open source spatial audio project optimized for mobile and desktop computing, any platform or software development tool provider can easily integrate with Resonance Audio. More cross-platform and tooling support means more distribution opportunities for content creators, without the worry of investing in costly porting projects,” Eric Mauskopf, project manager for Google wrote in a post.

According to the company, spatial audio is necessary to providing a sense of presence within virtual reality and augmented reality worlds.

The open source project will include a reference implementation of YouTube’s Ambisonic-based spatial audio decoder, which is compatible with the Ambisonic format used across the industry. It will also feature encoding, sound field manipulation and decoding techniques, and head related transfer functions to achieve rich spatial audio. Additionally, Google will open its library of optimized DSP classes and functions.

In addition, it is being open sourced as a standalone library and associated engine plugins, VST plugin tutorial and examples.

Since its November launch, Google says Resonance Audio has been used in many applications such as Pixar’s Coco VR for Gear VR and Disney’s Star Wars Jedi Challenges app for Android and iOS.

Other ways Google has been trying to create a sense of presence in VR is through experiments with Light Fields. According to the company, light fields create a sense of presence by creating motion parallax and realistic textures and lighting.

“With light fields, nearby objects seem near to you—as you move your head, they appear to shift a lot. Far-away objects shift less and light reflects off objects differently, so you get a strong cue that you’re in a 3D space. And when viewed through a VR headset that supports positional tracking, light fields can enable some truly amazing VR experiences based on footage captured in the real world,” the team wrote in a post.

As part of its experiment, Google is releasing an app on Steam VR called “Welcome to Light Fields” to show the potential of this technology.


Microsoft Windows 10 Fall Creators Update SDK now available

Developers can start preparing their applications for the next update of Windows 10 with the newly available Windows 10 Fall Creators Update SDK. The SDK features new tools for building mixed reality experiences, modernizing applications for today’s workplace, and building and monetizing games and apps.

“Windows 10 Fall Creators Update provides a developer platform that is designed to inspire the creator in each of us – empowering developers to build applications that change the way people work, play and interact with devices. To truly fulfill this platform promise, I believe that our developer platform needs to be centered around people and their needs.  Technology should adapt and learn how to work with us,” Kevin Gallo, corporate vice president of the Windows developer platform, wrote in a post.

According to the company, the next wave of virtual and augmented reality is mixed reality. With Windows Mixed Reality, developers can create immersive experiences that are reusable across platforms and device form factors. “Windows 10 was designed from ground up for spatial interactions and the next wave in this journey is Windows Mixed Reality, uniting the digital and real world to create a rich, immersive world. As humans, we interact with space constantly, and Windows Mixed Reality will feel the most natural for users,” Gallo wrote.

To modernize apps for the workplace, the SDK enables developers to create and update existing apps with Visual Studio 2017 version 15.4, integration of .NET Standard 2.0, and an improved Windows 10 deployment system.

In addition, developers can build better game and app experiences with the Expanded Resources feature in the Fall Xbox One Update, the Xbox Live Creators Program, and the Mixer SDKs for major game engines and languages.

Gartner’s top 10 technology trends for 2018

With only a couple more months left of the year, Gartner is already looking ahead to the future. The organization announced its annual top strategic technology trends at the Gartner Symposium/ITxpo this week.

The basis of Gartner’s trends depends on whether or not they have the potential to disrupt the industry, and break out into something more impactful.

The top 10 strategic technology trends, according to Gartner, are:

    1. AI foundation: Last year, the organization included artificial intelligence and machine learning as its own trend on the list, but with AI and machine learning becoming more advance, Gartner is looking at how the technology will be integrated over the next five years. “AI techniques are evolving rapidly and organizations will need to invest significantly in skills, processes and tools to successfully exploit these techniques and build AI-enhanced systems,” said David Cearley, vice president and Gartner Fellow. “Investment areas can include data preparation, integration, algorithm and training methodology selection, and model creation. Multiple constituencies including data scientists, developers and business process owners will need to work together.”
    2. Intelligent apps and analytics: Continuing with its AI and machine learning theme, Gartner predicts new intelligent solutions that change the way people interact with systems, and transform the way they work.
    3. Intelligent things: Last in the AI technology trend area is intelligent things. According to Gartner, these go beyond rigid programming models and exploit AI to provide more advanced behaviors and interactions between people and their environment. Such solutions include: autonomous vehicles, robots and drones as well as the extension of existing Internet of Things solutions.
    4. Digital twin: A digital twin is a digital representation of real-world entities or systems, Gartner explains. “Over time, digital representations of virtually every aspect of our world will be connected dynamically with their real-world counterpart and with one another and infused with AI-based capabilities to enable advanced simulation, operation and analysis,” said Cearley. “City planners, digital marketers, healthcare professionals and industrial planners will all benefit from this long-term shift to the integrated digital twin world.”
    5. Cloud to the edge: Internet in the Internet of Things has brought up the notion of edge computing. According to Gartner, Edge computing is a form of computing topology that processes, collects and delivers information closer to its source. “When used as complementary concepts, cloud can be the style of computing used to create a service-oriented model and a centralized control and coordination structure with edge being used as a delivery style allowing for disconnected or distributed process execution of aspects of the cloud service,” said Cearley.
    6. Conversational platforms: Conversational platforms such as chatbots are transforming how humans interact with the emerging digital world. This new platform will be in the form of question and command experiences where a user asks a question and the platform is there able to respond.
    7. Immersive experience: In addition to conversational platforms, experiences such as virtual, augmented and mixed reality will also change how humans interact and perceive the world. Outside of video games and videos, businesses can use immersive experience to create real-life scenarios and apply it to design, training and visualization processes, according to Gartner.
    8. Blockchain: Once again, blockchains makes the list for its evolution into a digital transformation platform. In addition to the financial services industry, Gartner sees blockchains being used in a number of different apps such as government, healthcare, manufacturing, media distribution, identity verification, title registry, and supply chain.
    9. Event driven: New to this year’s list is the idea that the business is always looking for new digital business opportunities. “A key distinction of a digital business is that it’s event-centric, which means it’s always sensing, always ready and always learning,” saidYefim Natis, vice president, distinguished analyst and Gartner Fellow. “That’s why application leaders guiding a digital transformation initiative must make ‘event thinking’ the technical, organizational and cultural foundation of their strategy.”
    10. Continuous adaptive risk and trust: Lastly, the organization sees digital business initiatives adopting a continuous adaptive risk and trust assessment (CARTA) model as security becomes more important in a digital world. CARTA enables businesses to provide real-time, risk and trust-based decision making, according to Gartner.

“Gartner’s top 10 strategic technology trends for 2018 tie into the Intelligent Digital Mesh. The intelligent digital mesh is a foundation for future digital business and ecosystems,” said Cearley. “IT leaders must factor these technology trends into their innovation strategies or risk losing ground to those that do.”

To compare, last year’s trends are available here.

In addition, the organization also announced top predictions for IT organizations and users over the next couple of years. The predictions include: early adopters of visual and voice search will see an increase in digital commerce revenue by 30% by 2021; five of the top seven digital giants (Alibaba, Amazon, Apple, Baidu, Facebook, Google, Microsoft and Tencent) will willfully self-disrupt by 2020; and IoT technology will be in 95% of electronics by 2020.

HTC stock suspension adds fuel to Google acquisition rumors

HTC is to suspend trading of its stock on the TWSE tomorrow, ahead of what it says will be the release of “material information” pertaining to its business. The news was reported earlier by the FT. 

The move has spiked speculation about a potential sale of HTC’s mobile division, with the Taiwanese device maker struggling for years to try to turn around its fortunes in the fiercely competitive smartphone market.

In recent years loss-making quarters have become the norm for HTC, which posted its first ever loss making quarter in Q3 2013. And despite management changes, portfolio trimming and even pushing into a new product category (VR, via a partnership with games publisher Valve) it has been unable to pull its business out of a long slide.

Media reports in Asia have recently linked Google’s parent company Alphabet with a possible acquisition of HTC’s mobile business. And a note on HTC’s investor website references speculation in the China Times that “HTC might announce the sale to Google” — going on to specify its “countermeasure” to this report is to state: “HTC does not comment on market rumor or speculation”.

If Google is indeed set to pick up HTC’s smartphone division it would not be the first time it’s swooped in to try to salvage one of its Android OEMs. The company acquired Motorola Mobility in 2011, shelling out $12.5BN on the purchase. Then in 2014 it sold the division to Lenovo for $2.91BN — holding on to “the vast majority” of Motorola’s patent portfolio.

Smartphone leaker Evan Blass has tweeted that he’s been sent a copy of an internal HTC invite for employees to a town hall meeting tomorrow — which apparently includes “Google acquisition” as one of the topics.

[1] Someone sent me a copy of an internal invitation to an HTC employee Town Hall meeting tmrw (9/21). One alleged topic: Google acquisition


Nanome launches VR collaboration platform Matryx

Virtual reality developer Nanome has launched Matryx, an open-source platform that they hope will change how people collaborate to advance problem solving in STEM.

The three primary components of Matryx are the bounty system for rewarding users for solving problems posed on the platform by researchers, a library of digital assets for use in developing models and solutions and a marketplace for exchanging these ideas and assets.

In a company white paper, the developers outlined their inspiration in the Henri Clay Institute of Mathematics’ “Millennium Problems” and the ways they’re avoiding the problems with that challenge with collaborative, iterative rewards.

“Rather than simply establishing a goal and rewarding the first past the post, Matryx tracks the provenance of assets, enables low friction collaboration, and divides rewards amongst all participants,” reads the white paper.

Problem-solving on Matryx is incentivized with a blockchain-based bounty system that uses their custom MTX token, which is deployed as a smart contract on the Ethereum blockchain. The company will be putting the first of these tokens up for sale early next month.

“Our goal is to make Matryx the de facto standard for decentralized collaboration, proving that a global community of collaborators will yield innovation faster than work attempted in siloed teams,” said Steve McCloskey, CEO of Nanome. “Matryx aims to create an environment that encourages healthy competition and the open exchange of ideas for critical fields like math, science, technology and engineering. By leveraging blockchain-based bounties, we’re also able to reward contributors openly, encourage future collaboration, and evolve the platform as user needs demand.”

The platform works together with Nanome’s suite of VR STEM tools, which includes Calcflow, a mathematics problem-solving and modeling application, and nano-one, a VR interface for designing and simulating at nanoscale. The open-source license means anyone can make software to interface with the platform, and while their focus is currently only on STEM, Nanome says they can see Matryx evolving into other fields.

The impact of virtual and augmented reality on corporate developers

It was more than 30 years ago that Microsoft Windows was first released. At the time, it was a radical departure from the text-based interfaces that dominated most screens. It has been over 25 years since Windows 3.0, the first point that people started really paying attention to Windows. Suddenly, there was a reason for people to pay attention. Multitasking was important and something that DOS didn’t do. However, Windows had to fight off the perception that it was for games to find its footing as a useful productivity tool.

Fast forward to today, when virtual and augmented reality solutions are making fun games because of platforms like Oculus Rift and Pokémon Go. Games have thrust these technology solutions into the consciousness of individuals and business leaders who wonder how they can be used for productivity instead of entertainment. It’s up to today’s corporate developers to take the technologies and make them productive.

The Learning Curve
Like the learning curve for Windows decades ago, the learning curve for virtual and augmented reality isn’t shallow – but it’s a learning curve that corporate developers can overcome. While most corporate developers could, historically, safely ignore threading and performance concerns in their corporate applications, that is no longer the case. The need for real-time feedback creates a need to defer processing and focus on the interaction with the user. This means learning, improving your learning, or relearning how to manage threads in your applications.

It also means looking for optimal processing strategies that most developers haven’t seen since their computer science textbooks. With Moore’s Law creating massive processing capacity in both central processing capabilities as well as graphics capabilities, it’s been some time since most developers have needed to be concerned with which strategy was the fastest. However, as these platforms emerge, it’s necessary to revisit the quest for optimal processing strategies – including the deferral of work into background threads.

More challenging than development-oriented tasks may be the need to develop models in three-dimensional space. Most developers eventually got decent with image editors to create quick icons that designers could later replace. However, building 3D models is different. It means a different set of tooling and a different way of thinking.

The Applications
Most corporate developers were relegated to working on applications that were far removed from the reality of the day-to-day business. Recording the transactions, scanning the forms, tracking customer interactions… all were important, but disconnected from making the product, servicing the customer, or getting the goods to the end user. VR and AR are changing that. Instead of living in a world that’s disconnected from how the user does their work, VR and AR are integrating how users do their work and how they learn.

In the corporate world, VR applications include training with materials that are too expensive or dangerous to work with in reality – and the remote management of robots and drones that do the work that is too difficult for a human to do. Instead of controlling electrons in a computer, VR software is moving atoms or rewiring human brains. Training is no longer boring videos of someone else doing something, it’s an interactive simulation that used to be too expensive to build. The opportunity to remotely control through VR provides the benefits of human judgement with the safety of not exposing humans to dangerous conditions.

AR can augment humans. Instead of having to memorize reams of content, it can be displayed in-context. Knowledge management systems have traditionally been boring repositories of information that’s difficult to access. AR connects the knowledge repository with its use.

AR also makes accessible to humans sensors that are beyond our five senses. AR can bring thermal imaging, acoustic monitoring, and other sensors into our range of perception through visual or auditory cues. Consider how digital photography transformed the photography industry. Now everyone can get immediate feedback and can make adjustments instead of having to wait for the development process.

The Change
Ultimately, VR and AR mean that developers get the chance to have a greater and more tangible impact on the world around them. They can be a part of augmenting human capacity, reducing the risk to humans, and to improve training. All it takes is a refocus on threading, performance, and learning a bit about 3D modeling.

Intel has decided to retire its older chipset Atom x5-Z8100P SoC which has been available for years. Microsoft’s Hololens has also been using this chipset since its first launch. According to reports, Intel has asked the customers of this chipset to place their final order until September 30th.

While the orders will be shipped to them by October 30th. And after this deadline, there won’t be any production for this chipset by Intel. While Microsoft has been working on the next version of their Hololens Augmented Reality product. This news may indicate us that Microsoft may have included a new chipset for the next version of HoloLens.

Microsoft had already revealed that they would be using a newer Hololens Processing Unit HPU) 2.0 that will have an AI co-processor integrated into it. Microsoft Hololens’ next version is expected to come in mid-2018 and they will be ready with the consumer version of Hololens in 2019-2020.

Microsoft has many times reiterated that they consider Mixed reality and artificial intelligence as the future of computing. Microsoft had said at Computex last June that they hope to see 80 million VR devices in consumers’ hands by 2020. It looks Microsoft is well on track with their Mixed Reality vision.


To date, VR backpack PCs have been aimed at gamers who just don’t want to trip over cords while they’re fending off baddies. But what about pros who want to collaborate, or soldiers who want to train on a virtual battlefield? HP thinks it has a fix.

It’s launching the Z VR Backpack, a spin on the Omen backpack concept that targets the pro crowd. It’s not as ostentatious as the Omen, for a start, but the big deal is its suitability to the rigors of work.

The backpack is rugged enough to meet military-grade drop, dust and water resistance standards, and it uses business-class hardware that includes a vPro-enabled quad Core i7 and Quadro P5200 graphics with a hefty 16GB of video memory.


The wearable computer has tight integration with the HTC Vive Business Edition, but HP stresses that you’re not obligated to use it — it’ll work just fine with an Oculus Rift or whatever else your company prefers.

The pro parts do hike the price, though, as you’ll be spending at least $3,299 on the Z VR Backpack when it arrives in September. Not that cost is necessarily as much of an issue here — that money might be trivial compared to the cost of a design studio or a training environment.

There’s even a project in the works to showcase what’s possible. HP is partnering with a slew of companies (Autodesk, Epic Games, Fusion, HTC, Launch Forth and Technicolor) on a Mars Home Planet project that uses VR for around-the-world collaboration. Teams will use Autodesk tools to create infrastructure for a million-strong simulated Mars colony, ranging from whole buildings to pieces of clothing.

The hope is that VR will give you a better sense of what it’d be like to live on Mars, and help test concepts more effectively than you would staring at a screen. You can sign up for the first phase of the project today.


AltspaceVR, the social app that’s sort of like Second Life for the Gear VR, HTC Vive and Oculus Rift, “can’t afford to keep the virtual lights on” anymore.

The company has announced that it’s closing down the application on August 3rd, 10 PM Eastern, due to unforeseen financial difficulties. While its investors last funded the project back in 2015, the shutdown still came as a surprise since it already had a deal in the works.

Unfortunately, that deal suddenly fell through, and the team ran out of time to raise enough funds to keep the application running.

In the announcement post, the company revealed that the app only has around 35,000 monthly users. It says that’s “pretty good for the size of the VR market,” though, and if there’s anything that made its old investors pull back, it’s the general slowness of the market’s growth.

While the app itself will definitely be shuttered, AltspaceVR’s head honchos are still thinking of what to do with the company itself.

The social application served as a hangout for VR headset users, who held concerts and various events with their avatars — Engadget editor Sean Buckley even once played Dungeons & Dragons within its virtual confines.

Needless to say, people formed friendships in AltspaceVR’s virtual world, and the company wants to remind everyone to take the next few days to find new ways to connect.

The team will hold a VR party on August 3rd to give everyone a chance to say goodbye and will take the virtual universe offline for good when the clock strikes 10.

Following YouTube’s support for 4K live 360 video late last year, it was only a matter of time before Facebook caught up. After all, Zuckerberg’s betting big on VR.

Today, the social media giant announced the launch of 4K resolution for live 360 streams, meaning we can soon enjoy more immersive VR live streams via the Facebook 360 app for Gear VR. This also preps the platform well for when 4K VR headsets finally go mainstream.

As part of this update, Facebook is also bringing its Donate Button and Scheduled Live features over to its live 360 video service, in the hopes of driving 360 video viewership while helping out non-profits.

Also announced today is the expansion of Facebook’s “Live 360 Ready” program, which now covers both devices and software suites from third parties.

In addition to the Insta360 Air, Insta360 Nano and Insta360 Pro that were already certified earlier this year, the “Live 360 Ready” list now also includes Giroptic IO, Nokia Ozo, ION360U, ORAH 4I, Z Cam S1, 360fly 4K Pro and Garmin VIRB 360. You’ll be able to spot these in the shops thanks to the Facebook Live logo on their new packaging. Oddly enough, the better known brands like Samsung and Ricoh are missing here.

As for software, Facebook has certified the following for 360 video publishing: Assimilate SCRATCH VR, Groovy Gecko, LiveScale, Teradek, Voysys, Wowza and Z CAM WonderLive.

By finally bumping the quality of its live 360 videos and also helping creators pick the right tools, Facebook is now headed in the right direction.

That said, we’re still waiting for the next wave of VR headsets packing 4K display resolution, so that we can go deeper into the immersive content. Hopefully this is something that Facebook and others are already working hard on.