virtualreality

Auto Added by WPeMatico

This Unique Nikon Lens Can Capture 360-Degree Views With No Stitching

A designer duo has created a first-of-its-kind lens that can record 360-degree spherical video content that doesn’t need to be stitched in post-processing and can be used with any conventional camera.

Rob Englert and Meyer Giordano are two experienced industrial and interaction designers with a particular interest in augmented (AR) and virtual reality (VR) and have worked with brands such as Bose, Chobani, KODAK, RIDGID, and others. Together, they founded the (sphere) optics brand under which they developed the unique (sphere) Pro1 lens, which can capture everything in full 360-degree view, and creates shooting opportunities that otherwise would not be possible.

What makes it unique is that this lens eliminates the stitching process normally found in spherical or VR content production that combines the perspective of multiple cameras and lenses together. Additionally, creators can also use their existing camera and workflow with the Pro1 as opposed to needing wholly separate equipment.

The idea for the lens was born out of personal experiences after Englert lost his young brother. This life-changing moment happened long before everyone had smartphones in their pockets, which left Englert with almost no memories captured of his brother in a video format.

This fuelled his drive to use all of his skills as an industrial designer “to explore different ways to capture moments in time,” which can later be revisited over and over again. He says that he would “give almost anything for just a couple more minutes” with his brother, and although that is not possible anymore, he hopes it can be made possible for others in the future.

Lens Design

Originally, the novel lens was developed as part of the duo’s ongoing work on AR and VR technologies, with the inclusion of non-fungible tokens (NFTs) in the project’s funding process. Both designers were producing 360-degree videos, using multi-camera arrays in a housing that they had designed and 3D-printed. The process was arduous because the content was recorded across several cameras and the final output needed stitching. They separated each video into frames, combined each set of frames into a panorama, and then recompiled it as a video. Even then, the final product would still have knit lines visible where the views were merged.

Both designers spent a lot of time reviewing how this process could be improved, which is where the idea of a single lens and single-camera setup originated. Instead of using optical design software, they created potential shapes as 3D models and then simulated the result using regular 3D rendering software, which eventually, after some trial and error, gave them a potential physical prototype. They still had to test it in a real-life scenario, which meant the prototype had to be made by hand. Once finessed, they began working with optics professionals to further refine the design and get one step closer to a finished lens.

To cover 100-percent of the environment with the lens, they started with a regular circular fisheye lens with a field of view of approximately 180-degrees and mounted a mirror that reflects its image downward like a periscope in front of it. Then, using the cross-section of this setup, they extruded it in a circle around the axis of the mirror, which ended up with several donut-shaped lens elements around a cone-shaped mirror, which then provided the 180-degree vertical field of view of the original fisheye design as well as a 360-degree horizontal view from being revolved around the center.

The current design of the lens has 12 elements: one reflective — the mirror — and 11 refractive, including twi torus-shaped elements that surround the mirror. Most of the elements of this lens are just as unique as the design itself and cannot be found in any other existing lenses.

The parts are made from specialist engineering plastics — using a family of materials called cyclic olefin copolymers — in a process called single-point diamond turning, which is the only way to generate the complex aspheric forms that make the lens design possible. These plastics have similar optical properties to glass, but are much lighter and easier to shape, and are used in top-end scientific applications, like space telescopes.

The duo chose to go with Nikon F-mount because it’s easily adaptable to most other common standards, making it easier for content creators to use the equipment they already have instead of investing in a completely new system while maintaining full control over the image at all times. The single-lens construction also allows capturing content that a standard VR setup couldn’t due to space limitations.

The lens has a fixed f/8 aperture and uses a 1mm focal length. It is 150 millimeters (5.9 inches) wide and 198 millimeters (7.8 inches) long and weighs 1.8 kg (4 lbs).

Currently, these lenses are produced extremely slowly. They are fabricated one unit at a time and assembled by hand, which makes the individual cost quite high. However, with a large order, the duo says that costs could be significantly reduced by molding most of the elements. The two believe that this lens can benefit a wide range of industries from film, documentary, gaming, and entertainment, all the way to engineering, military, surveying, and more.

Photo and Video Footage

For stills, the lens produces a circular image with a black void in the center. The area near the center of the image corresponds to the forward end of the lens, and the outer periphery of the image circle is the direction facing the camera body. In their words, “It looks a lot like the black hole from ‘Interstellar.’”

Converted equirectangular of same image, as above, shot in Big Bear CA.
(sphere) Pro1 lens was used to help NASA document the James Webb Space Telescope at Goddard Space Flight Center as part of its journey to space.

The circular image can be used as-is if desired, but most 360-degree video players use the standard equirectangular format. For this reason, (sphere) provides an ST map, which is an image that acts as a positional lookup table to tell the computer how to rearrange the pixels. This is also commonly used to remove lens distortion and internally is similar to what Adobe Camera Raw’s distortion correction does. The duo says that Adobe could add specific support for this if they desired.

Designers have also created 3D meshes that can be inserted into game engines, such as Unity or Unreal, to unwrap the image in real-time, “which is very useful either to act as a monitor for recording or to facilitate live streaming to VR headsets.” Overall, the process of converting (sphere) videos to VR experiences is simple and can be done on iOS and Android devices, the designers claim.

Get Involved

To fund the project and the ongoing development, the team offers high resolution, fully immersive VR moments as limited edition NFTs on Mintable. These moments were filmed using the lens and buyers can use their VR headset to virtually enter the place and time captured. The purchase includes the video in standard equirectangular format at 8K resolution for viewing via VR headset, along with the native circular projection of the (sphere) Pro 1 lens at the native 4,048 x 4,048 resolution.

Additionally, the company offers a token that can be exchanged for a physical copy of the (sphere) Pro1 lens as more units become available. You can also view a (sphere) gallery with more image samples here.

Kandao’s Obsidian Pro 12K 360 Camera is Loaded with 8 APS-C Sensors

Kandao is going all-in on high-quality, VR-ready, 360-degree video with its newly-announced Obsidian Pro. The company is calling it the “world’s-first cinematic 360-degree camera with 8 APS-C sensors” to offer 24 million pixels of high-resolution 12K capture.

As noted by Fstoppers, Kandao announced the camera on April 15 and says it can capture 12K by 12K video at up to 30 frames per second in 12-bit RAW (DNG8) or 10-bit All-I 4:2:2. Instead of each of the eight cameras needing its own storage system, the Kandao is able to send all the footage to what it describes as an 8-in-1 SSD that comes in 4, 8, and 16 terabyte configurations. For reference, a 16TB SSD can hold up to 90 minutes of RAW video and 100 minutes of All-I footage.

The Obsidian Pro camera weighs 11,160 grams (~24.6 pounds) and has a build size of 369.2 × 369.2 × 236.8mm (~14.5 x 14.5 x 9.3 inches).

Each of the 8 cameras offers 130 degrees of horizontal field of view and 195 degrees of vertical field of view. When overlapped, the eigh-lens structure “ensures that every single angle can be covered by three lenses so that capturing cover can reach 300%.” Each lens array is constructed of 14 elements in 10 groups with a triplet of achromatic lenses and 18 layers of anti-reflective coating.

The Obsidian Pro also has a built-in “nine-axis gyroscope” that the company promises to precisely synchronize and achieve “excellent image stability” and eliminated rolling shutter.

The large APS-C sensors are a significant upgrade from the Micro Four Thirds sensors available in the next closest competitor, the Insta360 Titan. With them, Kandao says each of the 8 cameras has 14 stops of dynamic range. The company also claims that it is the first panoramic camera that is equipped with lenses that have adjustable apertures from f/2.8 to f/16. As such, it has up to 16 stops of adjustable exposure that Kandao says will allow for precise control of depth of field and for creative motion blur capture.

The camera also offers adjustable electronic focus control between “near” and “normal” as the two options:

Near
Normal

The camera can be controlled both from the Kandao Obsidian Pro App or via the LCD touchscreen located on the Obsidian Pro device itself. It supports 10-gigabit ethernet as well as Wi-Fi 6 and Bluetooth 5 and also comes equipped with a built-in GPS module.

For audio, the Obsidian Pro has a standard 3.5mm audio input as well as a USB audio device input on top of its built-in omnidirectional microphones to realize full directional sound around the camera array. The camera system can be powered by a wall outlet or with a V-mount battery using a battery power supply module.

Currently, all models of the Obsidian Pro camera are on backorder, but the “Standard” version retails for $24,500 which does not appear to come with an SSD. Adding a 4TB SSD increases the price to $26,700, and currently the 8TB version comes with a free upgrade to 16TB for $29,000. When the free upgrade is no longer available, the 16TB version will cost $35,000. Those prices include shipping cost and shipping insurance, but does not cover VAT or customs expenses.

The Tamagotchi Pix Mixes the Classic Virtual Pet with a Built-in Camera

The latest Tamagotchi model, Tamagotchi Pix, adds a touch of augmented reality with a built-in camera that allows players to take photos with their digital pet along with a few other additional gameplay features.

Tamagotchi virtual and pocket-sized pet game consoles were popular in the 1990s but since then the brand’s popularity dipped as technology improved and as other options like smartphones and augmented reality games such as Pokémon Go have taken off.

However, Japanese-brand Bandai made a comeback with its Tamagotchi On model, which came out in 2019 and was the first to feature a Bluetooth connection that it used to allow a transfer of virtual pets into the Tamagotchi On app, meet other characters, and more. The company reports, after the launch of this model, they noticed that “a whole new generation of kids fall in love with Tamagotchi,” which was a catalyst for further development.

After 25 years and 82 million units worldwide, Bandai launched its newest model, appropriately titled Tamagotchi Pix, which has added a built-in camera and improved the gameplay. Tamagotchi Pix still sports the classic features of a virtual pet-rearing but the added camera allows players to take photos with the device and add their virtual pets next to them. The Verge explains that the photos are then stored locally on the device and appear in a feed that documents the user’s gameplay time with their pet.

However, the image-sharing function is limited as Tamagotchi Pix can’t connect to Wi-Fi. If players want to share images with other users, they will need to take a picture of a “Tame Code” which, similar to a QR code, contains the entirety of the data you are sharing with others. The company plans to incorporate this in further gameplay by awarding Tame Codes for in-game items starting in August.

New camera functions aside, Tamagotchi Pix has replaced the familiar physical buttons around the screen with a touch-based system and has expanded gameplay features. For example, now the way Tamagotchi is raised and as well as their environment — such as room decorations — will affect their future profession and will determine what other eggs players can raise after their Tamagotchi has fully grown. Pets can also partake in new activities, such as painting, cooking, and even ordering food for delivery.

Although the new features cannot compete in the augmented reality realm with more advanced games available on the market, it still brings a touch of modern technology back into this retro childhood toy, introduces it to younger generations, and makes the player feel that their virtual pet is closer to being a part of their reality even if it’s just through a photograph of the two of them together. Engadget reports that players can also find up to 100 Tamagotchi buddies as they explore the world, which resembles a Pokemon Go-inspired gameplay.

The new model comes shaped like a hatching egg, with the eggshell on top and the screen and the buttons below. It is available in four colors: “floral” (pink), “sky” (purple), “ocean” (blue), and “nature” (green). Pink and purple models are readily available but the blue and green are yet to be released. The presale price for Tamagotchi Pix is $60.

Future Apple VR Products Include Headset, Glasses, and Contacts: Report

The mixed reality headset that rumors have previously stated as in development is due for release by 2022, a new report states. Apple isn’t stopping there, however, as augmented reality glasses are set to launch by 2025 and contact lenses by the next decade.

In a research note with TF International Securities — a Hong Kong-based analyst firm that provides financial institutions with asset management services and advice — obtained by Macrumors, well-regarded analyst Ming-Chi Kuo writes that Apple’s mixed-reality/augmented reality roadmap prediction includes three phases: first a helmet-type headset by 2022, a glasses-style device by 2025, and a contact lens device between 2030 and 2040.

“We foresee that the helmet product will provide AR and VR experiences, while glasses and contact lens types of products are more likely to focus on AR applications,” Kuo writes.

Kuo also notes that the expected price for the headset will be $1000, one-third the price that was rumored in February. He also says that the current prototypes of the headset weigh somewhere between 200 and 300 grams, but Apple is trying to reduce that by 100 to 200 grams for the final design to make it much lighter than many existing VR/AR headsets on the market, hence the high price.

Kuo adds that it will be equipped with Sony’s micro-OLED displays and external optical modules to enable the augmented reality experience, but can also operate as a VR headset. This echoes the rumor published by The Intercept in February. It should be portable and have its own computing power and storage but isn’t designed to be a mobile device like an iPhone or iPad.

The reported glasses and contact lenses are part of the roadmap that Kuo predicts, but it’s unclear how the analyst is coming to those conclusions. The “Apple Glasses” would provide an “optical see-through AR experience” and would be positioned more like a mobile product, which is similar to the now-defunct Google Glass. The contact lenses report is part of Kuo’s far-future prediction and provided few details.

While the Google AR/VR headset has not been more than a few rumors, Kuo’s report, which is designed to inform investors, lends further credence to Apple’s foray into the field.

(via MacRumors)


Image credits: Photo licensed through Deposit Photos.

Apple is Making a $3,000 Dual 8K Mixed-Reality Headset: Report

There have been rumors that Apple was going to enter the virtual reality/augmented reality hardware space, but a recent report cites the most detail that has been revealed thus far: more than a dozen cameras, dual 8K displays, and a $3,000 price.

The Information, the source of this latest rumor, has cited its latest report based on images of the headset that its reporters viewed as well as details it says it received from a source within Apple with “direct knowledge” of the device. According to that source, the publication believes the device is in the later stages of development and could come to market as early as 2022.

The report cites a “late-stage prototype” that looks like a traditional VR headset that fully encloses the wearer’s eyes and blocks out the outside world, which would put them in a fully immersive virtual world presented to the wearer through dual 8K displays. Additionally, it appears to have the ability to project virtual items onto real space, a la augmented reality.

Those more than a dozen cameras are supposedly used to track eye movements as well as hand gestures, and combined with LiDAR sensors it should also be able to accurately measure distances between the wearer and real-world objects. Apple also plans to integrate its spatial audio technology into the interchangeable headbands so that wearers will be able to experience more realistic 3D-audio in the virtual worlds. The headset is likely to be powered by custom Apple silicon.

Rumors of an Apple virtual reality headset have been in the news for years. Dual 8K displays, for example, were rumored as far back as 2018 so these reports aren’t wholly unexpected.

The $3,000 price seems to put the device up against the Microsoft Hololens 2, a $3,500 business-oriented headset which indicates Apple might not be targeting the average consumer with their device. Still, mixed reality headsets for enterprise customers can get significantly more expensive, like Canon’s $38,400 headset. Given that Apple is pushing the limits of pricing on its other devices, it’s less clear that this reported $3,000 price would really limit it to business customers.

(via Engadget)


Image credits: Header image by Stella Jacob.

Canon to Release $38,400 ‘Mixed Reality’ Headset

Canon is set to release the MREAL S1 headset in late February for 4 million yen (~$38,400) that can synthesize and display CG images and place them into the wearer’s real-life location to create a mixed/augmented reality experience.

The MREAL S1 is a video see-through head-mounted display and works by using two forward-facing CMOS sensors to “see” the real world so that the headset can properly render virtual objects into the 3D space and project that combination towards the headset’s wearer.

In a promotional video for the product shown above, Canon points to an example use case where a person would wear the headset in order to interact with a concept vehicle and see it as it would be if it were actually in a space. The headset can take a 3D model or CG render and properly size it and place it in 3D space so that it can be seen from all angles.

While clearly objects can’t be felt, they can be interacted with. Other examples shown indicate that parts could be virtually placed but sit alongside objects that actually exist. This “mixed reality” interface would best work to better visualize new construction, fill out empty rooms to see furniture placement, or allow factories to look at how new production lines would be arranged and experience the workability and flow of those lines.

Theoretically, by being able to confirm full-scale CG objects with direct reference to reality, the MREAL S1 would be able to reduce costs by improving communication efficiency and reducing the number of trials needed to reach a finished design.

This is not the first time Canon has released a headset like the MREAL S1. In the video, Canon references the MD-10 and MD-20, which are previous generation headsets by Canon that are significantly larger and heavier.

The price, as mentioned, doesn’t come cheap. The approximately $38,400 headset is being marketed mainly to corporations and includes the unit, software, computer, and the cost of maintaining the whole device (both hardware and software).

(via DC Watch via Canon Watch)

These Free Steam Games Let You Roam Realistic Locations with a Camera

Australian indie game developer Matt Newell has released a new free Steam game called Castle Rock Beach, West Australia. It allows you to freely explore a realistic recreation of Australia’s southwest coast with a camera.

Just check out these screenshots for a taste of how beautiful and photorealistic the immersive world is:

Here’s a 2-minute video showing what gameplay looks like:

The game allows you to explore the world at your own pace while completing casual photography objectives with the in-game DSLR camera.

What’s impressive is that Newell is a full-time engineering student at university who’s doing this game development in his free time. Newell shares that he had experience in photography and color before he started learning to build with Unreal Engine in February 2018.

Newell built the game over 9 months (again, in his free time) by bringing in photoscanned models (of things like plants, rocks, trees), working on lighting/color, and adding interactive elements.

This latest game joins a growing list of locations Newell has released already.

In May 2020, Newell released Wakamarina Valley, New Zealand, which is set in the idyllic forested landscape of the Wakamarina Valley, located near Queen Charlotte Sound on New Zealand’s South Island.

In March 2020, Newell released Mýrdalssandur, Iceland, which focuses on Iceland’s southern coast.

Newell’s first release was in October 2019. Titled Explore Fushimi Inari, the game is a realistic recreation of the renowned Japanese shrine, Fushimi Inari Taisha.

All four games are available on Steam and have VR headset compatibility (though compatibility may cost you a little money). The basic West Australia, Iceland, and Japan games are free, while the New Zealand game costs $4.