Auto Added by WPeMatico

Dual Camera Platform Tries to Fix the Issues with 360-Degree Cameras

Researchers have designed a new, dual camera platform with the aim of making up for the poor resolution output that comes with most 360-degree cameras.

360-degree field of view cameras of varying types and price ranges have been available in consumer and specialist security markets for some time now. They are often used for virtual business tours, real estate, security, sports and action, travel, and other purposes.

However, a recent study, titled “A Hybrid Camera System for High-Resolutinaztion of Target Objects in Omnidirectional Images” conducted by researchers Chinthaka Premachandra and Masaya Tamaki has pointed out that there is a need to address the lower resolution problem that comes with these types of omnidirectional 360-degree field-of-view cameras, especially in regard to surveillance cameras where poor resolution makes it difficult to identify distant objects.

The study explains that with the use of a fisheye lens, which collects light across a wider range, a 360-degree camera is a cost-effective solution to surveillance that leaves hardly any to no blind spots, compared to using several general-purpose cameras to collectively cover the same field of view.

Image by Chinthaka Premachandra and Masaya Tamaki, Creative Commons

One solution to the low resolution problem would be to increase video resolution to at least 4K, as proposed by a 2015 study by Budagavi et al., which brings an additional set of complexities, such as the need for efficient compression technologies that can handle increased bitrates as a result.

Premachandra and Tamaki also agreed to the need to increase video resolutions to at least 4K “in order to mitigate the problem of resolution degradation due to wide fields of view when using omnidirectional cameras for monitoring,” but the practicalities of detecting and tracking objects, combined with converting such video into 2D images is a complex, costly, and lengthy process.

For example, PetaPixel recently reported on (sphere) Pro1 360-degree lens which can capture video content with no stitching but this type of technology comes at a cost that can be out of reach for most.

This is why both researchers came to the idea of designing a system that takes images from a conventional omnidirectional camera and at the same time also uses a separate camera that can capture high-resolution images of objects further away, whereby combined, the system would enable better identification of moving objects while still affordable.

In the study, the duo created a prototype hybrid camera platform that consists of one omnidirectional camera and two pan-tilt (PT) cameras with a 180-degree field-of-view on either side. When an indistinct target region is detected using the 360-degree camera, the PT camera is then used to capture a high-resolution image of the target.

Connections between each camera and the computer. Image by Chinthaka Premachandra and Masaya Tamaki, Creative Commons.

The two used Raspberry Pi Cameras on which a pan-tilt module was mounted and then connected to the system through a Raspberry Pi 3 Model B. Then, all of the parts of the setup were connected to a personal computer to allow overall control.

The camera platform used in the study. Image by Chinthaka Premachandra and Masaya Tamaki, Creative Commons.

“The researchers first processed an omnidirectional image to extract a target region, following which its coordinate information was converted into angle information (pan and tilt angles) and subsequently transferred to the Raspberry Pi,” Science Daily explains.

Following the experiments, the study concluded that this type of system did indeed deliver higher-resolution images compared to those that were generated from a single 360-camera. However, one issue that arose was a possible time delay in the process. For example, when a moving object is determined as a target to be captured with a high-resolution image, there is a potential shift because it takes a moment for the appropriate PT camera to capture it.

As a potential countermeasure, the duo proposes the Kalman filtering technique, which is an algorithm that gives estimates of unknown variables which in this case are future coordinates of the moving object, which would counteract the shift encountered.

Science Daily reports that Premachandra is confident that their proposed camera system “will create positive impacts on future applications employing omnidirectional imaging such as robotics, security systems, and monitoring systems.”

The full published study, including details of all experiments, can be read on the IEEE Xplore website.

Image credits: Header image by Chinthaka Premachandra and Masaya Tamakiused and used under Creative Commons.

This Unique Nikon Lens Can Capture 360-Degree Views With No Stitching

A designer duo has created a first-of-its-kind lens that can record 360-degree spherical video content that doesn’t need to be stitched in post-processing and can be used with any conventional camera.

Rob Englert and Meyer Giordano are two experienced industrial and interaction designers with a particular interest in augmented (AR) and virtual reality (VR) and have worked with brands such as Bose, Chobani, KODAK, RIDGID, and others. Together, they founded the (sphere) optics brand under which they developed the unique (sphere) Pro1 lens, which can capture everything in full 360-degree view, and creates shooting opportunities that otherwise would not be possible.

What makes it unique is that this lens eliminates the stitching process normally found in spherical or VR content production that combines the perspective of multiple cameras and lenses together. Additionally, creators can also use their existing camera and workflow with the Pro1 as opposed to needing wholly separate equipment.

The idea for the lens was born out of personal experiences after Englert lost his young brother. This life-changing moment happened long before everyone had smartphones in their pockets, which left Englert with almost no memories captured of his brother in a video format.

This fuelled his drive to use all of his skills as an industrial designer “to explore different ways to capture moments in time,” which can later be revisited over and over again. He says that he would “give almost anything for just a couple more minutes” with his brother, and although that is not possible anymore, he hopes it can be made possible for others in the future.

Lens Design

Originally, the novel lens was developed as part of the duo’s ongoing work on AR and VR technologies, with the inclusion of non-fungible tokens (NFTs) in the project’s funding process. Both designers were producing 360-degree videos, using multi-camera arrays in a housing that they had designed and 3D-printed. The process was arduous because the content was recorded across several cameras and the final output needed stitching. They separated each video into frames, combined each set of frames into a panorama, and then recompiled it as a video. Even then, the final product would still have knit lines visible where the views were merged.

Both designers spent a lot of time reviewing how this process could be improved, which is where the idea of a single lens and single-camera setup originated. Instead of using optical design software, they created potential shapes as 3D models and then simulated the result using regular 3D rendering software, which eventually, after some trial and error, gave them a potential physical prototype. They still had to test it in a real-life scenario, which meant the prototype had to be made by hand. Once finessed, they began working with optics professionals to further refine the design and get one step closer to a finished lens.

To cover 100-percent of the environment with the lens, they started with a regular circular fisheye lens with a field of view of approximately 180-degrees and mounted a mirror that reflects its image downward like a periscope in front of it. Then, using the cross-section of this setup, they extruded it in a circle around the axis of the mirror, which ended up with several donut-shaped lens elements around a cone-shaped mirror, which then provided the 180-degree vertical field of view of the original fisheye design as well as a 360-degree horizontal view from being revolved around the center.

The current design of the lens has 12 elements: one reflective — the mirror — and 11 refractive, including twi torus-shaped elements that surround the mirror. Most of the elements of this lens are just as unique as the design itself and cannot be found in any other existing lenses.

The parts are made from specialist engineering plastics — using a family of materials called cyclic olefin copolymers — in a process called single-point diamond turning, which is the only way to generate the complex aspheric forms that make the lens design possible. These plastics have similar optical properties to glass, but are much lighter and easier to shape, and are used in top-end scientific applications, like space telescopes.

The duo chose to go with Nikon F-mount because it’s easily adaptable to most other common standards, making it easier for content creators to use the equipment they already have instead of investing in a completely new system while maintaining full control over the image at all times. The single-lens construction also allows capturing content that a standard VR setup couldn’t due to space limitations.

The lens has a fixed f/8 aperture and uses a 1mm focal length. It is 150 millimeters (5.9 inches) wide and 198 millimeters (7.8 inches) long and weighs 1.8 kg (4 lbs).

Currently, these lenses are produced extremely slowly. They are fabricated one unit at a time and assembled by hand, which makes the individual cost quite high. However, with a large order, the duo says that costs could be significantly reduced by molding most of the elements. The two believe that this lens can benefit a wide range of industries from film, documentary, gaming, and entertainment, all the way to engineering, military, surveying, and more.

Photo and Video Footage

For stills, the lens produces a circular image with a black void in the center. The area near the center of the image corresponds to the forward end of the lens, and the outer periphery of the image circle is the direction facing the camera body. In their words, “It looks a lot like the black hole from ‘Interstellar.’”

Converted equirectangular of same image, as above, shot in Big Bear CA.
(sphere) Pro1 lens was used to help NASA document the James Webb Space Telescope at Goddard Space Flight Center as part of its journey to space.

The circular image can be used as-is if desired, but most 360-degree video players use the standard equirectangular format. For this reason, (sphere) provides an ST map, which is an image that acts as a positional lookup table to tell the computer how to rearrange the pixels. This is also commonly used to remove lens distortion and internally is similar to what Adobe Camera Raw’s distortion correction does. The duo says that Adobe could add specific support for this if they desired.

Designers have also created 3D meshes that can be inserted into game engines, such as Unity or Unreal, to unwrap the image in real-time, “which is very useful either to act as a monitor for recording or to facilitate live streaming to VR headsets.” Overall, the process of converting (sphere) videos to VR experiences is simple and can be done on iOS and Android devices, the designers claim.

Get Involved

To fund the project and the ongoing development, the team offers high resolution, fully immersive VR moments as limited edition NFTs on Mintable. These moments were filmed using the lens and buyers can use their VR headset to virtually enter the place and time captured. The purchase includes the video in standard equirectangular format at 8K resolution for viewing via VR headset, along with the native circular projection of the (sphere) Pro 1 lens at the native 4,048 x 4,048 resolution.

Additionally, the company offers a token that can be exchanged for a physical copy of the (sphere) Pro1 lens as more units become available. You can also view a (sphere) gallery with more image samples here.