augmentedreality

Auto Added by WPeMatico

This Unique Nikon Lens Can Capture 360-Degree Views With No Stitching

A designer duo has created a first-of-its-kind lens that can record 360-degree spherical video content that doesn’t need to be stitched in post-processing and can be used with any conventional camera.

Rob Englert and Meyer Giordano are two experienced industrial and interaction designers with a particular interest in augmented (AR) and virtual reality (VR) and have worked with brands such as Bose, Chobani, KODAK, RIDGID, and others. Together, they founded the (sphere) optics brand under which they developed the unique (sphere) Pro1 lens, which can capture everything in full 360-degree view, and creates shooting opportunities that otherwise would not be possible.

What makes it unique is that this lens eliminates the stitching process normally found in spherical or VR content production that combines the perspective of multiple cameras and lenses together. Additionally, creators can also use their existing camera and workflow with the Pro1 as opposed to needing wholly separate equipment.

The idea for the lens was born out of personal experiences after Englert lost his young brother. This life-changing moment happened long before everyone had smartphones in their pockets, which left Englert with almost no memories captured of his brother in a video format.

This fuelled his drive to use all of his skills as an industrial designer “to explore different ways to capture moments in time,” which can later be revisited over and over again. He says that he would “give almost anything for just a couple more minutes” with his brother, and although that is not possible anymore, he hopes it can be made possible for others in the future.

Lens Design

Originally, the novel lens was developed as part of the duo’s ongoing work on AR and VR technologies, with the inclusion of non-fungible tokens (NFTs) in the project’s funding process. Both designers were producing 360-degree videos, using multi-camera arrays in a housing that they had designed and 3D-printed. The process was arduous because the content was recorded across several cameras and the final output needed stitching. They separated each video into frames, combined each set of frames into a panorama, and then recompiled it as a video. Even then, the final product would still have knit lines visible where the views were merged.

Both designers spent a lot of time reviewing how this process could be improved, which is where the idea of a single lens and single-camera setup originated. Instead of using optical design software, they created potential shapes as 3D models and then simulated the result using regular 3D rendering software, which eventually, after some trial and error, gave them a potential physical prototype. They still had to test it in a real-life scenario, which meant the prototype had to be made by hand. Once finessed, they began working with optics professionals to further refine the design and get one step closer to a finished lens.

To cover 100-percent of the environment with the lens, they started with a regular circular fisheye lens with a field of view of approximately 180-degrees and mounted a mirror that reflects its image downward like a periscope in front of it. Then, using the cross-section of this setup, they extruded it in a circle around the axis of the mirror, which ended up with several donut-shaped lens elements around a cone-shaped mirror, which then provided the 180-degree vertical field of view of the original fisheye design as well as a 360-degree horizontal view from being revolved around the center.

The current design of the lens has 12 elements: one reflective — the mirror — and 11 refractive, including twi torus-shaped elements that surround the mirror. Most of the elements of this lens are just as unique as the design itself and cannot be found in any other existing lenses.

The parts are made from specialist engineering plastics — using a family of materials called cyclic olefin copolymers — in a process called single-point diamond turning, which is the only way to generate the complex aspheric forms that make the lens design possible. These plastics have similar optical properties to glass, but are much lighter and easier to shape, and are used in top-end scientific applications, like space telescopes.

The duo chose to go with Nikon F-mount because it’s easily adaptable to most other common standards, making it easier for content creators to use the equipment they already have instead of investing in a completely new system while maintaining full control over the image at all times. The single-lens construction also allows capturing content that a standard VR setup couldn’t due to space limitations.

The lens has a fixed f/8 aperture and uses a 1mm focal length. It is 150 millimeters (5.9 inches) wide and 198 millimeters (7.8 inches) long and weighs 1.8 kg (4 lbs).

Currently, these lenses are produced extremely slowly. They are fabricated one unit at a time and assembled by hand, which makes the individual cost quite high. However, with a large order, the duo says that costs could be significantly reduced by molding most of the elements. The two believe that this lens can benefit a wide range of industries from film, documentary, gaming, and entertainment, all the way to engineering, military, surveying, and more.

Photo and Video Footage

For stills, the lens produces a circular image with a black void in the center. The area near the center of the image corresponds to the forward end of the lens, and the outer periphery of the image circle is the direction facing the camera body. In their words, “It looks a lot like the black hole from ‘Interstellar.’”

Converted equirectangular of same image, as above, shot in Big Bear CA.
(sphere) Pro1 lens was used to help NASA document the James Webb Space Telescope at Goddard Space Flight Center as part of its journey to space.

The circular image can be used as-is if desired, but most 360-degree video players use the standard equirectangular format. For this reason, (sphere) provides an ST map, which is an image that acts as a positional lookup table to tell the computer how to rearrange the pixels. This is also commonly used to remove lens distortion and internally is similar to what Adobe Camera Raw’s distortion correction does. The duo says that Adobe could add specific support for this if they desired.

Designers have also created 3D meshes that can be inserted into game engines, such as Unity or Unreal, to unwrap the image in real-time, “which is very useful either to act as a monitor for recording or to facilitate live streaming to VR headsets.” Overall, the process of converting (sphere) videos to VR experiences is simple and can be done on iOS and Android devices, the designers claim.

Get Involved

To fund the project and the ongoing development, the team offers high resolution, fully immersive VR moments as limited edition NFTs on Mintable. These moments were filmed using the lens and buyers can use their VR headset to virtually enter the place and time captured. The purchase includes the video in standard equirectangular format at 8K resolution for viewing via VR headset, along with the native circular projection of the (sphere) Pro 1 lens at the native 4,048 x 4,048 resolution.

Additionally, the company offers a token that can be exchanged for a physical copy of the (sphere) Pro1 lens as more units become available. You can also view a (sphere) gallery with more image samples here.

The Tamagotchi Pix Mixes the Classic Virtual Pet with a Built-in Camera

The latest Tamagotchi model, Tamagotchi Pix, adds a touch of augmented reality with a built-in camera that allows players to take photos with their digital pet along with a few other additional gameplay features.

Tamagotchi virtual and pocket-sized pet game consoles were popular in the 1990s but since then the brand’s popularity dipped as technology improved and as other options like smartphones and augmented reality games such as Pokémon Go have taken off.

However, Japanese-brand Bandai made a comeback with its Tamagotchi On model, which came out in 2019 and was the first to feature a Bluetooth connection that it used to allow a transfer of virtual pets into the Tamagotchi On app, meet other characters, and more. The company reports, after the launch of this model, they noticed that “a whole new generation of kids fall in love with Tamagotchi,” which was a catalyst for further development.

After 25 years and 82 million units worldwide, Bandai launched its newest model, appropriately titled Tamagotchi Pix, which has added a built-in camera and improved the gameplay. Tamagotchi Pix still sports the classic features of a virtual pet-rearing but the added camera allows players to take photos with the device and add their virtual pets next to them. The Verge explains that the photos are then stored locally on the device and appear in a feed that documents the user’s gameplay time with their pet.

However, the image-sharing function is limited as Tamagotchi Pix can’t connect to Wi-Fi. If players want to share images with other users, they will need to take a picture of a “Tame Code” which, similar to a QR code, contains the entirety of the data you are sharing with others. The company plans to incorporate this in further gameplay by awarding Tame Codes for in-game items starting in August.

New camera functions aside, Tamagotchi Pix has replaced the familiar physical buttons around the screen with a touch-based system and has expanded gameplay features. For example, now the way Tamagotchi is raised and as well as their environment — such as room decorations — will affect their future profession and will determine what other eggs players can raise after their Tamagotchi has fully grown. Pets can also partake in new activities, such as painting, cooking, and even ordering food for delivery.

Although the new features cannot compete in the augmented reality realm with more advanced games available on the market, it still brings a touch of modern technology back into this retro childhood toy, introduces it to younger generations, and makes the player feel that their virtual pet is closer to being a part of their reality even if it’s just through a photograph of the two of them together. Engadget reports that players can also find up to 100 Tamagotchi buddies as they explore the world, which resembles a Pokemon Go-inspired gameplay.

The new model comes shaped like a hatching egg, with the eggshell on top and the screen and the buttons below. It is available in four colors: “floral” (pink), “sky” (purple), “ocean” (blue), and “nature” (green). Pink and purple models are readily available but the blue and green are yet to be released. The presale price for Tamagotchi Pix is $60.

Snap is Working on AR Glasses, Selfie Drone: Report

Snap is working on its next-generation glasses that will be able to layer Snapchat lenses (which are augmented reality effects) onto the environment without needing to use a smartphone camera, according to a new report. It has also reportedly revived its plans to introduce a selfie-drone.

According to a report from The Information, and noted by Engadget, the next generation camera glasses aren’t actually being developed with the intention of deployment to the average consumer. Instead, the company is focusing efforts on developers and creators, those who currently are responsible for making the app’s more popular SnapChat AR filters.

Snapchat is reportedly gearing up to officially announce these new Spectacles in May during its developer conference. Theoretically, if the glasses are successful, Snap could aim to launch more widely.

While Snap is most well-known for its social media app, the company has successfully brought hardware to market. Snap launched the Snap Spectacles 3 in 2019 and still offers them for sale today. The original Snap spectacles were a viral hit in 2016 when they launched in special pop-up vending machines in select cities. Eventually, the Spectacles became avaialble to order on Amazon. In 2018, Snap launched its second-generation Spectacles that allowed you to capture photos using the glasses.

That’s not the only hardware Snap is investigating. The report goes on to say that the company is once again looking into producing a selfie drone. In 2017, Snap reportedly acquired the LA-based drone company Ctrl Me Robotics before looking at another China-based acquisition, Zero Zero Robotics. Back then, The Information reported that Zero Zero reached out to Snap for funding help, which was later denied by the robotics company’s CEO.

Since then, news about Snap and drones has been largely pretty quiet, but that is apparently set to change. Snap supposedly did eventually invest in Zero Zero to the amount of $20 million, but it did not outright acquire the company. Since then, the new report indicates that Snap’s own engineers have been working on developing an in-house product. It’s not clear when such a drone would come to market and how it would differentiate itself in what has become a very crowded space.

Future Apple VR Products Include Headset, Glasses, and Contacts: Report

The mixed reality headset that rumors have previously stated as in development is due for release by 2022, a new report states. Apple isn’t stopping there, however, as augmented reality glasses are set to launch by 2025 and contact lenses by the next decade.

In a research note with TF International Securities — a Hong Kong-based analyst firm that provides financial institutions with asset management services and advice — obtained by Macrumors, well-regarded analyst Ming-Chi Kuo writes that Apple’s mixed-reality/augmented reality roadmap prediction includes three phases: first a helmet-type headset by 2022, a glasses-style device by 2025, and a contact lens device between 2030 and 2040.

“We foresee that the helmet product will provide AR and VR experiences, while glasses and contact lens types of products are more likely to focus on AR applications,” Kuo writes.

Kuo also notes that the expected price for the headset will be $1000, one-third the price that was rumored in February. He also says that the current prototypes of the headset weigh somewhere between 200 and 300 grams, but Apple is trying to reduce that by 100 to 200 grams for the final design to make it much lighter than many existing VR/AR headsets on the market, hence the high price.

Kuo adds that it will be equipped with Sony’s micro-OLED displays and external optical modules to enable the augmented reality experience, but can also operate as a VR headset. This echoes the rumor published by The Intercept in February. It should be portable and have its own computing power and storage but isn’t designed to be a mobile device like an iPhone or iPad.

The reported glasses and contact lenses are part of the roadmap that Kuo predicts, but it’s unclear how the analyst is coming to those conclusions. The “Apple Glasses” would provide an “optical see-through AR experience” and would be positioned more like a mobile product, which is similar to the now-defunct Google Glass. The contact lenses report is part of Kuo’s far-future prediction and provided few details.

While the Google AR/VR headset has not been more than a few rumors, Kuo’s report, which is designed to inform investors, lends further credence to Apple’s foray into the field.

(via MacRumors)


Image credits: Photo licensed through Deposit Photos.

Apple is Making a $3,000 Dual 8K Mixed-Reality Headset: Report

There have been rumors that Apple was going to enter the virtual reality/augmented reality hardware space, but a recent report cites the most detail that has been revealed thus far: more than a dozen cameras, dual 8K displays, and a $3,000 price.

The Information, the source of this latest rumor, has cited its latest report based on images of the headset that its reporters viewed as well as details it says it received from a source within Apple with “direct knowledge” of the device. According to that source, the publication believes the device is in the later stages of development and could come to market as early as 2022.

The report cites a “late-stage prototype” that looks like a traditional VR headset that fully encloses the wearer’s eyes and blocks out the outside world, which would put them in a fully immersive virtual world presented to the wearer through dual 8K displays. Additionally, it appears to have the ability to project virtual items onto real space, a la augmented reality.

Those more than a dozen cameras are supposedly used to track eye movements as well as hand gestures, and combined with LiDAR sensors it should also be able to accurately measure distances between the wearer and real-world objects. Apple also plans to integrate its spatial audio technology into the interchangeable headbands so that wearers will be able to experience more realistic 3D-audio in the virtual worlds. The headset is likely to be powered by custom Apple silicon.

Rumors of an Apple virtual reality headset have been in the news for years. Dual 8K displays, for example, were rumored as far back as 2018 so these reports aren’t wholly unexpected.

The $3,000 price seems to put the device up against the Microsoft Hololens 2, a $3,500 business-oriented headset which indicates Apple might not be targeting the average consumer with their device. Still, mixed reality headsets for enterprise customers can get significantly more expensive, like Canon’s $38,400 headset. Given that Apple is pushing the limits of pricing on its other devices, it’s less clear that this reported $3,000 price would really limit it to business customers.

(via Engadget)


Image credits: Header image by Stella Jacob.

Canon to Release $38,400 ‘Mixed Reality’ Headset

Canon is set to release the MREAL S1 headset in late February for 4 million yen (~$38,400) that can synthesize and display CG images and place them into the wearer’s real-life location to create a mixed/augmented reality experience.

The MREAL S1 is a video see-through head-mounted display and works by using two forward-facing CMOS sensors to “see” the real world so that the headset can properly render virtual objects into the 3D space and project that combination towards the headset’s wearer.

In a promotional video for the product shown above, Canon points to an example use case where a person would wear the headset in order to interact with a concept vehicle and see it as it would be if it were actually in a space. The headset can take a 3D model or CG render and properly size it and place it in 3D space so that it can be seen from all angles.

While clearly objects can’t be felt, they can be interacted with. Other examples shown indicate that parts could be virtually placed but sit alongside objects that actually exist. This “mixed reality” interface would best work to better visualize new construction, fill out empty rooms to see furniture placement, or allow factories to look at how new production lines would be arranged and experience the workability and flow of those lines.

Theoretically, by being able to confirm full-scale CG objects with direct reference to reality, the MREAL S1 would be able to reduce costs by improving communication efficiency and reducing the number of trials needed to reach a finished design.

This is not the first time Canon has released a headset like the MREAL S1. In the video, Canon references the MD-10 and MD-20, which are previous generation headsets by Canon that are significantly larger and heavier.

The price, as mentioned, doesn’t come cheap. The approximately $38,400 headset is being marketed mainly to corporations and includes the unit, software, computer, and the cost of maintaining the whole device (both hardware and software).

(via DC Watch via Canon Watch)