ARcore Platform Overview
The purpose of the ARcore platform is to provide a customer with a complete system integration to create his own high performance AR product based on the ARcore.
The platform consists of two parts:
The zero-latency FPGA module, managing the camera pass-through and overlay mixing. Besides the ARcore IP implemented on the FPGA, an ARM dual-core CPU runs a firmware that allows the user to control all aspects of the zero-latency pipeline. Firmware functions can be accessed via a Linux-based API.
The GPU module runs a graphics engine (e.g. Unity) to provide the AR content to be superimposed on the camera image. Furthermore we provide a middleware with support algorithms (QR marker detection, visual SLAM etc.) for common high level image processing tasks to further increase development efficiency for the user.
The two modules exchange information via a PCIe and HDMI interface. Cameras and inertial measurement unit are connected directly to the FPGA module. The GPU module connects to an external server to stream data to/from other HMDs via wireless LAN.
ARcore Prototype System
The core components of the ARcore platform are implemented in our existing prototype system. The FPGA SoC module is based on an Arria10 development kit extended with custom hardware.
The GPU component is equipped with a special HDMI-to-CSI interface to create an HMDI input for the Jetson TX1 platform. The raw camera image is forwarded from the imaging pipeline on the FPGA. A resulting content overlay image is returned back to the imaging via an HDMI output. In the future this exchange will happen using PCIe.
The prototype uses a modified virtual reality head mounted display. The cameras of the HMD are directly connected to the main platform system via LVDS lines for lowest latency signal propagation.
Software Eats the World
The FPGA module as well as the GPU module both contain a software stack and API to be used by the developer. All important core functionality of the system is accessible through the software library and allow easy implementation of applications.