Xperia xz1 3d scanner

Sony Xperia XZ1 Boasts 3D Scanning Capabilities « Next Reality

By Tommy Palladino
8/31/17 8:47 AM

At IFA in Berlin today, Sony introduced the latest models of the Xperia line, with new features in tow that should appeal to creators of augmented and virtual reality content.

The Xperia XZ1 and XZ1 Compact both carry the Motion-Eye camera that can capture video at 960 frames per second for slow motion video, anticipate action or smiles to take a photo proactively, and continually focus on subjects as they move.

The headline feature for Next Reality readers, though, is the 3D Creator feature. Using a proprietary algorithm, 3D Creator offers users the ability to scan 3D objects in one minute. 3D Creator offers four modes – head scan, face scan, food scan, and freeform scan – each with their own guides to assist the user in scanning and after effects to modify the results.

3D Creator enables users to scan subjects in one of four modes (including head mode) and use the resulting models to print in 3D. Image by Sony/YouTube

Users can share the scans as stickers in messaging apps or upload their creations to sites like Sketchfab. The scans can be used in camera AR effects, live wallpapers, and third-party apps. Of course, they can also be replicated via 3D printers.

3D Creator will also recommend apps that take advantage of 3D models in Google Play; one could surmise that these recommended apps could include ARCore apps fairly soon.

Don't Miss: Scandy Gives Tango Owners a Taste of 3D Scanning

In recent years, some smartphone makers have attempted to appeal to specific user communities with niche features. For instance, the Tango platform gives users depth-sensing features for augmented reality experiences. The LG V20 found itself on a number of "best of" lists with advanced camera and audio recording capabilities that appealed to creators.

With its 3D Creator feature, Sony appears to appeal to a similar subset, namely virtual and augmented reality content creators or 3D printing enthusiasts.

(1) Sony Xperia XZ1 comes in four colors, including Moonlit Blue, (2) and White Silver. Images via Sony

Don't Miss: Qualcomm and Himax Align to Advance AR Hardware with 3D Camera Sensor

Running on the Qualcomm Snapdragon 835, the Xperia ZX1, with its 5.2 inch display, and its Compact cousin, with a 4.6 inch screen, offer a cadre of other features as well, including high-fidelity audio output, quick charging, and water resistance. Available worldwide next month, both phones will ship with Android 8.0 Oreo.

In addition, Sony revealed the Xperia XA1 Plus, which features a higher-capacity battery and a 23MP camera, but lacks the 3D Creator and other features of the ZX1 models. The XA1 Plus will be available in Fall 2017.

Follow Next Reality on Facebook, Twitter, Instagram, YouTube, and Flipboard
Sign up for Next Reality's daily newsletter

Cover image via Sony

Hot
Latest

How To: Improve Your Pokémon's Stats in Pokémon GO
News: Medical Software Maker Medivis Closes $2.
3 Million in Funding to Bring HoloLens to Surgical Platform
News: Nike Begins Using Augmented Reality for Sneaker Fitting in Mobile App & in Retail Stores
News: BMW Opens Augmented Reality Showroom for Plug-In Hybrids on the Web via 8th Wall
AR Snapshots: Live Out Your Sci-Fi Dreams with Iron Man Suits & Lightsabers in Snapchat
HoloLens Dev 101: How to Use Coroutines to Add Effects to Holograms
News: Facebook & Ray-Ban Stories Smartglasses, Details on the Specs, Colors, Prices, Software, and Availability
Market Reality: Microsoft Intros HoloLens 2 Variant, Canon Sets High Price for AR Headset & Apple AR Rumors Re-Emerge
News: The 50 Best Augmented Reality Apps for iPhone, iPad & Android Devices
News: Zara AR App Brings Virtual Models to Life in Stores
News: Magic Leap Lawsuit vs.
Nreal Takes Major Step Forward After Legal Team Scours the Planet in Search of Elusive Founder
News: Niantic Preps Multiplayer Mode for Pokémon GO, Plans Limited Launch of Real World Platform via Creator Program in 2020
News: Camera IQ Gives Cosmetics Brands the Power to Push Virtual Try-on Effects to Snapchat, Facebook, & Instagram
ARKit 101: How to Place 2D Images, Like a Painting or Photo, on a Wall in Augmented Reality
ARKit 101: How to Measure Distance Between Two Points on a Horizontal Plane in Augmented Reality
Meta 2 Dev 101: How to Get Started Developing for the Meta 2 in Unity
News: Here's How Anyone Can Create an AR Snow Globe Scene Using Adobe Aero & Some Holiday Imagination
News: HoloLens 2, All the Specs — These Are the Technical Details Driving Microsoft's Next Foray into Augmented Reality
News: HTC Opens Up Augmented Reality Development for HTC Vive
News: Apple Pushes ARKit Closer to the AR Cloud with Location Anchors, Expands Face Tracking Support
All Features

How To: Improve Your Pokémon's Stats in Pokémon GO
News: Nike Begins Using Augmented Reality for Sneaker Fitting in Mobile App & in Retail Stores
News: Medical Software Maker Medivis Closes $2.
3 Million in Funding to Bring HoloLens to Surgical Platform
News: BMW Opens Augmented Reality Showroom for Plug-In Hybrids on the Web via 8th Wall
AR Snapshots: Live Out Your Sci-Fi Dreams with Iron Man Suits & Lightsabers in Snapchat
HoloLens Dev 101: How to Use Coroutines to Add Effects to Holograms
News: Facebook & Ray-Ban Stories Smartglasses, Details on the Specs, Colors, Prices, Software, and Availability
Market Reality: Microsoft Intros HoloLens 2 Variant, Canon Sets High Price for AR Headset & Apple AR Rumors Re-Emerge
News: The 50 Best Augmented Reality Apps for iPhone, iPad & Android Devices
News: Magic Leap Lawsuit vs.
Nreal Takes Major Step Forward After Legal Team Scours the Planet in Search of Elusive Founder
News: Zara AR App Brings Virtual Models to Life in Stores
News: Camera IQ Gives Cosmetics Brands the Power to Push Virtual Try-on Effects to Snapchat, Facebook, & Instagram
News: Niantic Preps Multiplayer Mode for Pokémon GO, Plans Limited Launch of Real World Platform via Creator Program in 2020
ARKit 101: How to Place 2D Images, Like a Painting or Photo, on a Wall in Augmented Reality
ARKit 101: How to Measure Distance Between Two Points on a Horizontal Plane in Augmented Reality
News: Here's How Anyone Can Create an AR Snow Globe Scene Using Adobe Aero & Some Holiday Imagination
Meta 2 Dev 101: How to Get Started Developing for the Meta 2 in Unity
News: HoloLens 2, All the Specs — These Are the Technical Details Driving Microsoft's Next Foray into Augmented Reality
News: HTC Opens Up Augmented Reality Development for HTC Vive
News: Apple Pushes ARKit Closer to the AR Cloud with Location Anchors, Expands Face Tracking Support
All Hot Posts

Sony Xperia XZ1 and XZ1 Compact with 3D scanning cameras: Hands on

Mobile

Android

It's just too bad it's not 2013 when 3D printing was the hot new thing.

All products featured here are independently selected by our editors and writers. If you buy something through links on our site, Mashable may earn an affiliate commission.

> Tech > Mobile

Time has not been kind to Sony. While the Japanese electronics company enjoys great success with its PlayStation business, the number of people who'd wait in line to get its new Xperia smartphones has all but evaporated.

At IFA 2017 in Berlin, the company announced a new pair of Android smartphones, the Xperia XZ1 and XZ1 Compact, that show it hasn’t really learned anything in the last few years.

SEE ALSO: 6 reasons why you should care about the Galaxy Note 8 even if you own an iPhone

As competitors switch to bezel-less and edge-to-edge displays (even no-brands like Doogee are doing it) -- with and without notches -- and include wireless charging, Sony continues to fail to launch new phones even with fingerprint sensors or other forms of biometric security such as iris scanning or face recognition in the U. S. In 2017, that’s just unacceptable.

Still no fingerprint sensors in Sony's new Xperia XZ1 phones...for the U.S. models. Credit: lili sams/mashable

Sony’s Xperia phones used to be pretty damn cutting-edge and competitive, tech-wise, with iPhones and whatever Galaxy phone Samsung was touting. And while the internal specs on the new XZ1 and XZ1 Compact are there -- it has a Qualcomm Snapdragon 835 chip, 4GB of RAM, 64GB of storage (32GB on the Compact), and a microSD card slot -- the rest of the hardware is underwhelming.

All of the colors the XZ1 will be available in Credit: LILI SAMS/MASHABLE

Let’s talk design, because, frankly, I think they look dated. Sony’s stuck with its “loop surface and seamless metal” industrial design. Don’t get me wrong, the phones look very Sony, but the huge forehead and chin bezels above the and below the screens aren’t going to age well, especially with the iPhone 8 around the corner.

And speaking of the screens, they’re are on the small side and just average-looking. Sony used to go overkill with 4K displays in its phones, but the XZ1 only has a 5.2-inch 1080p full HD display and the XZ1 Compact has a smaller 4.6-inch 720p screen.

The 5.2-inch Xperia XZ1 (left) and 4.6-inch XZ1 Compact (right) Credit: LILI SAMS/MASHABLE

The XZ1’s screen is HDR-ready, and whenever you watch HDR content from places like Amazon, Netflix, and YouTube, it’ll automatically display more dynamic range, so you'll get a better, more accurate picture. But when most everybody has (or will) moved to longer 18:9 or 18.5:9 displays and embraced OLED for the deepest blacks and most poppin’ colors, Sony is still pushing sub-Quad HD resolution screens with a 16:9 aspect ratio. I’m sure there are tons of people out there that’ll appreciate the Compact’s smaller size, but 720p in 2017 for a premium phone? Ehh, no thanks.

Battery life is also peculiar. Both phones have the same 2,700 mAh battery. That’s a smaller capacity than just about every flagship phone, most of which have 3,000 mAh or greater. Granted, the smaller screens and lower resolutions don’t require as much power, but still. I’m concerned at how long these phones will last. Sony’s Compact phones are known for their stamina -- the Z3 Compact had a battery that could last up to three days -- so I am hopeful.

These shortcomings may be frustrating for people who want the best there is, but the new camera features really shows how out of touch Sony is with mobile trends.

No dual cameras on Sony's XZ1 Credit: LILI SAMS/MASHABLE

Rather than dual cameras -- a feature every flagship phone is embracing in some way -- the XZ1 comes with a single 19-megapixel camera. The “Motion Eye” camera as Sony calls it has all of the jargon-named shooting features like pre-focusing and ridiculous 960 fps (frames per second) slow-motion recording introduced in previous Xperia phones.

New on the XZ1, however, is a 3D-scanning feature. Sony says users can scan four types of objects in under 60 seconds. Instead of spending thousands on a 3D scanner, you could use a special pre-loaded app to scan 3D objects and then send them to a 3D printer or third-party service to print out.

This would have been nice in 2013... Credit: LILI SAMS/MASHABLE

This would have been a genius idea -- a one-stop shop for 3D scanning and printing -- if it was 2013 and 3D printing was still hailed as the next big thing. But it’s 2017 and consumer interest in 3D printers has long evaporated. Even well-known consumer 3D printing brands like MakerBot had to sell themselves in order to survive.

3D scanning a face or head requires steady hands and a lot of patience Credit: LILI SAMS/MASHABLE

Now, to be fair, the 3D scanning is pretty decent. There are four scanning modes: one for faces, entire heads, food, and freeform objects. The 3D scanning process is a bit tricky, requires a lot of patience (for the scanner and another person if you’re doing heads and faces), and the right background situations (translation: spaces without lots of crap in it).

Scanning a fruit bowl Credit: LILI SAMS/MASHABLE

Looking over the 3D fruit bowl scan Credit: LILI SAMS/MASHABLE

During my brief time with the XZ1, I was able to get a couple of 3D scans, but not without some hiccups. Mashable photo editor Lili Sams grew impatient and annoyed as I followed the on-screen arrow markers indicating which direction to move the phone around her head for a proper scan. Confusing directions mid-scan also made me want to quit a few times, too.

After a few scan fails, we managed to get a few decent 3D scans of our faces and a bowl of fruit on a table, but the results were far from perfect. If you want the best 3D scan, you’ll definitely be investing more than 60 seconds per scan.

I know, I’m poo-pooing all over Sony’s new phones even though I haven’t thoroughly tested them, but that’s only because it’s so painful to see Sony stumble year after year. Sony used to be the world’s largest and most revered electronics company, but it keeps missing the mark in mobile. It’s unfortunate the new Xperia XZ1 and XZ1 Compact continues that tradition.

Opens in a new tab

Credit: Sony

Check out the new Xperia phones (opens in a new tab) (opens in a new tab)

The simplest 3D scanner on the example of a smartphone Sony Xperia XZ1 Compact

I share my experience of 3D scanning and printing of busts (souvenirs).
Until now, 3D scanners have been an expensive and inaccessible way to digitize, sometimes it's easier to draw a simple model by hand. But what if you need to create a bust or a model of a real object? This is where various tricks come to the rescue in the form of digitizing from photographs.
But in modern flagship smartphones, such functions have long been built in. The review will focus on the 3D Creator application for Sony Xperia.

Hello everyone!
Quite by accident I found out that there is a fairly good and inexpensive way for 3D scanning.
Specifically, the 3D Creator application for the Sony Xperia XZ1.

The 3D Creator app is free to download, but without a Sony Xperia smartphone it won't be much use. The program is a 3D model builder that allows you to make a 3D model with relatively low detail and post-processing from a series of shots from a good rear camera.
I found a relatively good Sony Xperia XZ1 Compact smartphone, which is equipped with a laser sensor suitable for creating 3D models.

Features:
Brand: Sony
Model: Xperia XZ1 Compact
Processor: Qualcomm Snapdragon 835, GPU Adreno 540
NFC: Present
OS: Android 8.0
Protection: IP65/IP68
Display: 4.6" IPS Triluminos Display Corning Gorilla Glass 5
Screen Resolution: 1280x720
Rear camera: 19 MP (Sony G Lens wide-angle optics) EIS stabilization, laser and phase detection autofocus, slo-mo super slow motion 960 fps.
Aperture f/2.0 Sony IMX400 Motion Eye
Front: 8 MP Sony IMX219PQ (Ultra wide angle lens up to 120°)
Battery: 2700mAh supporting QC3.0
Interfaces: BT 5.0, aptX HD, USB Type-C, Wi-Fi a/b/g/n/ac dual-band, 3G, 4G LTE
Additional features:: 3D scanning application

So, the peculiarity of this phone is the built-in laser rangefinder for autofocus, which is adapted for 3D scanning in the proprietary 3D Creator application. Scanning involves an accelerometer, a laser range finder and a rear camera.

There are a number of preset modes (portrait, bust, object scanning), as well as built-in processing (elimination of noise and defects) and post-processing (rotation, cropping, etc.). The result is *.mesh and *.stl for a regular 3D printer.

The result of the scan is extremely dependent on the lighting, the background (and dynamic objects in the background), the stability and accuracy of the scan itself. The result is not always satisfactory.

Built-in processing reduces noise, reduces detail and heavily blurs objects.

The scanning process is simple - select ~~victim~~ object, select a background (light, monotonous, if a person is on a chair, then in an empty room or against a clean wall).

Preset modes have their own scanning patterns - focus marks, direction of movement, etc.

As a matter of fact, the scanning process itself is not complicated - just rotate the smartphone along the pattern, trying to maintain the radius, uniformity, etc. The more accurate you do and the smoother the lighting, the better the result. It doesn’t always work out the first time, but there is an opportunity to “rescan”.

The result looks creepy - the application imposes a photo texture on the 3D model. It turns out so still a cabinet of curiosities.

Initially, the application was positioned as a "hype" for creating animations and 3D collages on Instagram and other social networks. With the ability to order a print in the store. But… the application allows you to export the original *.mesh model, as well as get a ready-made *.stl-file for printing.

When exporting, we get an archive with scan results.
File contents: *.mesh point cloud and textures in jpg.

But there is a separate option to generate a *.stl file that can be loaded into any 3D printer slicer. The screenshot shows the result of a bust scan. The printer prompts him to print 1:1.

The result of printing on a photopolymer printer

Detailing, of course, so-so. Will go for paint.

Such figures are suitable, for example, as a souvenir caricature, for example, if you paint / decorate a figure, for example, with a wig.

Or “make fun” by painting it creepily for Halloween

Complex or small objects, for example, a bolt thread, the smartphone does not scan - there is too much noise, which is then cut out by the application. As well as it is impossible to make high detailing.
A smartphone/3D Creator app is suitable for creating simple firukas, busts, commemorative models with low detail. Often the resulting models require "refinement" - the removal of scanning errors. Although, if you provide conditions and “fill” your hand, then there will be fewer such errors.
Thus, you can get a very simple (and inexpensive) hobby 3D scanner, the scanning itself does not require special skills or software, and the process itself is quite fast. Shooting a bust takes an average of 2-3 minutes of leisurely movement. It can be faster if you get used to it. Printing models on a photopolymer printer took 2 hours.

As for the application, it did not start on my other smartphones. Editing *.mesh is still a pleasure, as well as editing stl.
Looking ahead, I’ll say that Samsung also has a similar application and scanner, but 3D Creator on Xperia works much better. Perhaps in the future I will compare these functions.

Link to the application and description of 3D Creator on the official website.

How to create realistic 3D models with the Sony Xperia XZ1 Compact / Sudo Null IT News0001

As you already know, one of the highlights of the Sony Xperia XZ1 and Xperia XZ1 Compact smartphones is the Motion Eye camera. But it is interesting not only because of the high-quality shooting of photos and videos, but also because of the innovative 3D Creator function. We have already written about the capabilities of the camera itself, and today we will talk about what can be created in the 3D Creator application using the Sony Xperia XZ1 Compact as an example.

So, the 3D Creator function is available inside the application of the same name. In Russian firmware, it is simply called "3D Model Constructor" and is available on the XZ1 Compact right out of the box. And before you get started, we recommend that you update this app via Google Play as it is constantly being improved.

3D head and face scan

The application contains four basic modes: face scan, head scan, food scan and freeform scan. The last mode is recommended only for advanced users, so first we will deal with food, head and face.

At the very beginning, it is recommended to scan the face - this mode is used as a calibration, plus in the process you will understand exactly how the scanning takes place. And it happens simply. You simply move the device in the specified direction while the smartphone collects data on the physical movement of the smartphone in space, and the camera "scans" the texture of the face. The 3D model of the face looks specific, like the back half of the head and neck is cut vertically, but this model is used where a 360-degree rotation is not required. For example, when creating three-dimensional animated avatars for further use in augmented reality mode. And also to save 3D stickers that you can save it and send to a friend in any messenger.

But don't worry, a 360-degree head scan mode is also provided. True, before moving on to it, you will first have to scan the face. The resulting model can be uploaded, for example, to Sketchfab and then viewed on any device.

3D food and freeform scanning

While your friends are posting food photos online, you can take it to the next level and post a 3D model of a cake or a bowl of soup. The modes of scanning food and arbitrary shapes are separated, and in the first case, the smartphone recognizes typical shapes and uses blanks when building 3D models - this greatly simplifies and speeds up the process.

It is recommended to switch to working with arbitrary forms with a certain skill. But we can immediately give some useful tips. Try to make sure that the subject is evenly lit - if it is under a table lamp, move it to the side to remove hard shadows. And if it takes place on the street, then it is better to move into the shade - the result will be better. And, of course, do not demand the impossible from the application. 3D Creator is unlikely to be able to model in detail every needle on the back of a hedgehog, but with a cactus in a pot, a smartphone will have no problems.

Augmented reality, animation and 3D printing

The augmented reality photography mode allows you to use not only pre-made 3D models, but also your own 3D avatars. All this will be available for use when shooting in the standard AR effect application.

Not many features, though, and it's not the only application that takes advantage of 3D scanning capabilities.

Xperia xz1 3d scanner

Sony Xperia XZ1 Boasts 3D Scanning Capabilities « Next Reality

How To: Improve Your Pokémon's Stats in Pokémon GO

News: Medical Software Maker Medivis Closes $2.

News: Nike Begins Using Augmented Reality for Sneaker Fitting in Mobile App & in Retail Stores

News: BMW Opens Augmented Reality Showroom for Plug-In Hybrids on the Web via 8th Wall

AR Snapshots: Live Out Your Sci-Fi Dreams with Iron Man Suits & Lightsabers in Snapchat

HoloLens Dev 101: How to Use Coroutines to Add Effects to Holograms

News: Facebook & Ray-Ban Stories Smartglasses, Details on the Specs, Colors, Prices, Software, and Availability

Market Reality: Microsoft Intros HoloLens 2 Variant, Canon Sets High Price for AR Headset & Apple AR Rumors Re-Emerge

News: The 50 Best Augmented Reality Apps for iPhone, iPad & Android Devices

News: Zara AR App Brings Virtual Models to Life in Stores

News: Magic Leap Lawsuit vs.

News: Niantic Preps Multiplayer Mode for Pokémon GO, Plans Limited Launch of Real World Platform via Creator Program in 2020

News: Camera IQ Gives Cosmetics Brands the Power to Push Virtual Try-on Effects to Snapchat, Facebook, & Instagram

ARKit 101: How to Place 2D Images, Like a Painting or Photo, on a Wall in Augmented Reality

ARKit 101: How to Measure Distance Between Two Points on a Horizontal Plane in Augmented Reality

Meta 2 Dev 101: How to Get Started Developing for the Meta 2 in Unity

News: Here's How Anyone Can Create an AR Snow Globe Scene Using Adobe Aero & Some Holiday Imagination

News: HoloLens 2, All the Specs — These Are the Technical Details Driving Microsoft's Next Foray into Augmented Reality

News: HTC Opens Up Augmented Reality Development for HTC Vive

News: Apple Pushes ARKit Closer to the AR Cloud with Location Anchors, Expands Face Tracking Support