Sony scanner 3d

3D scanning: how Sony is driving innovative techniques

(Image credit: Sony)

3D scanning is evolving and Sony is leading the way. The field of photogrammetry – capturing high-definition 3D renders of objects, buildings and people, using images stitched together – has taken great leaps forward in recent years in line with camera technologies and computing capabilities.

There’s no doubt that it now constitutes its own specialist discipline. From data capture for video gaming assets to virtual production environments for film, TV and XR, the potential applications of photogrammetry are endless. And with the technical advances, we’re seeing the rise of digital humans: accurate or modified renders of real people that can be used in all manner of settings.

It's becoming easier too, we have a guide to the best portable 3D scanners available now and you can get some very affordable laptops for 3D modelling too. If you're new to this way of creating digital art, then take a look at how tutorial on how to create vintage art from 3D scans.

In recent years, Sony Digital Imaging has been making waves in the photogrammetry arena by supplying cutting-edge camera technology that allows their customers to capture renders with a level of realism previously unseen in 3D scanning. The manufacturer prides itself on building deep technical relationships throughout the implementation process that deliver startling results.

3D scanning: Peris Digital

Spanish firm Peris Digital is one of Sony's biggest clients, the studio's work has been seen in Netflix hit Bridgerton (Image credit: Sony / Peris Digital)

Sony’s roster of photogrammetry clients includes the Spanish firm Peris Digital. It’s a child company of Peris Costumes, who have been costume designers and curators since 1856.

Peris create digital scans of any garment, material or texture from any period, borrowing from their parent company’s archive of over seven million examples, for use in TV and film. Their clothing models have featured in film and TV productions for producers such as Netflix and Apple to name just two. Recent screen credits include the movie Dune, the TV series Bridgerton, La Casa de Papel (Money Heist), and Emily in Paris.

Peris have chosen to pursue photogrammetry over CGI because it guarantees the highest level of quality and agility needed to deliver visual FX. The optimal quality achieved through photogrammetry allows assets to be used on forefront characters, and not just with background scenes and extras.

Photogrammetry instead CGI produces more accurate and detailed models (Image credit: Sony / Peris Digital)

Peris uses Nvidia software at all stages of their product creation, as well as best-in-class Nvidia computers to handle their VFX. Peris featured as a keynote exhibitor in Nvidia's GTC event in November last year.

When it comes to VFX, Peris is an increasingly major player because of their high-end technology and their extensive experience in movie production. The company is also looking to offer the services of their photogrammetry rig to the gaming industry, but for now, VFX for TV and film is their central offering.

Creating a character for screen from scratch with costumes is a very time-consuming process, but Peris Digital can achieve remarkable results in just two-three hours by scanning existing materials and garments. To achieve all this, they use a main rig composed of 144 Sony Alpha 7C and 7R IV cameras, along with Sony’s G Master lenses, to generate 3D models ready to use in post-production work. They also have a smaller mobile rig which they can transport around the world to be used on-set.

Materials and costumes can be scanned in under three hours (Image credit: Sony / Peris Digital)

Garments can have an animated usage – both traditional animation or motion capture – or can be used with crowd software. Garments, accessories and props are available immediately and remotely, and Peris’ clients can access their clothing catalogue with no issues around stock shortage.

One of the challenges that Peris Digital faces is retopology – which is rebuilding a scan after the 2D images have been captured. Peris' artists must inform the software what kind of material it’s looking at, whether it’s cloth, metal and so on. This is a manual job at the moment, but the Peris team is working on automatic texture recognition that in future will do this job unaided.

Sony have played a critical role in helping Peris achieve its goals, beyond just supplying cameras

Resolution is the real key to what Peris delivers. The company is able to replicate the unique texture of garments to make the end results truly lifelike – an important advantage that photogrammetry has over conventional CGI. The Alpha 7R IV is Sony’s highest resolution model achieving 61MP, but the Alpha 7C is also the smallest and highest resolution in its category – and size really does matter on a rig with 144 cameras.

Sony has played a critical role in helping Peris achieve its goals, beyond just supplying cameras. In particular, Sony has helped extensively with integration of the cameras into the rig, and the vital component of camera synchronisation. Sony’s Camera Remote SDK is pivotal in delivering this. Sony also provide the G Master lenses (including the 135mm f1.8) that achieve the crystal-clear images needed for Peris’ work.

3D scanning: Stasis Media

Stasis Media specialises in creating accurate digital humans (Image credit: Sony / Peris Digital)

Sony has also worked closely with digital production agency Stasis Media, based in Plymouth, UK. Stasis Media specialises in digital human renders, where pixel-perfect images are needed to capture people that look truly real.

Each Stasis Media shoot now employs a number of Sony’s high-end Alpha 7R IV cameras to capture the intricate details of the face, and a handful of the smaller Sony RX0 II cameras to capture the body. All of the cameras are controlled and triggered via software to ensure that firing is virtually synchronous, and match the movements of the turntable, or the flaring of the LED lights. This is critical to enable the later process of accurately stitching the images together.

In the world of 3D scanning, one of the biggest challenges facing teams at the cutting edge is time

Craig Mason, director at Stasis Media says: “When you're trying to do a multi-camera shoot, it's really important that you manage to get the flashes synchronised with the camera exposures. We've found, particularly with the Alpha 7R IVs, that the synchronisation between multiple cameras and the flash exposures was very, very tight."

He continues: "In the world of 3D scanning, one of the biggest challenges facing teams at the cutting edge is time. Says Mason: “We were recently in Hamburg, creating 360-degree renders to help people learn how to cut and colour hair. The shoot was being undertaken in tandem with a very high-pressured video and photo collection being run at the same time, so we had just 60 to 120 seconds."

Mason adds: “When you've got a lot of cameras, it's very important to be able to coordinate the camera settings through an SDK like Sony’s. ”

3D scanning: Lidar Lounge

Lidar Lounge creates scans of objects and people for companies including Warner Bros, Disney and Netflix (Image credit: Sony / Lidar Lounge)

One final customer of Sony Digital Imaging in photogrammetry has been the UK-based Lidar Lounge. Since 2014, Lidar Lounge has been in the business of creating digital humans, but have a strong track record as well in set and location scanning and vehicle and object scanning. The studio works for film and TV, gaming and virtual reality, and architecture and design clients.

The outfit offers a highly bespoke service encompassing all aspects of visual FX and scanning. Their client roster is impressive, ranging from large film studios (including Warner Bros, Disney, Universal and Netflix) through British institutions such as the British Museum, the National Portrait Gallery and the National Theatre, and on to large consumer brands that include Dyson, Intel and AEG.

Lidar Lounge’s fruitful relationship with Sony is likely to continue well into the future

Lidar Lounge have scanned a myriad of items in the British museum, after hours, including Egyptian mummies and the iconic Rosetta Stone. Normally these sorts of exhibits are only on view to the public behind glass protection, but the company was allowed to see them privately, up-close in order to create immaculate 3D scans of them.

Lidar Lounge’s fruitful relationship with Sony is likely to continue well into the future, with numerous innovations and new techniques expected to be possible with what Sony provide. In the short term, the company will begin to incorporate Sony’s pixel shift functionality into their light passes. They’re also looking to use Sony’s Camera Remote SDK to create more precise trigger timings with multiple lighting setups that vary based on the cameras position within their rigs.

This article was originally published in 3D World Magazine. Read more excellent 3D content in 3D World, the world's best-selling magazine for CG artists. Subscribe to 3D World .

Read more:

The secrets of 3D scanning
Why 2D artists need to learn 3D
Scanning the future

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

Already have an account ? Sign in here

*Read 5 free articles per month without a subscription

Join now for unlimited access

Try first month for just £1 / $1 / €1

Already have an account ? Sign in here

Contact me with news and offers from other Future brandsReceive email from us on behalf of our trusted partners or sponsors

Rob Redman is the editor of 3D World and ImagineFX magazines and has a background in animation, visual effects, and photography. As a 3D artist he created the mothership in the Webby winning Plot Device and was animator on the follow-up; Order up. He has created training for Cinema 4D and Blackmagic Design Fusion artists. He's been a published product and food photographer since the age of 15. As well as being a multi-instrumentalist, Rob is also an avid beard grower.

Sony is Working on a 3D Scanner that will Allow Users to Put Real-World Items Into Video Games

By Peter Hunt Szpytek

Published Jan 7, 2022

Sony has expressed interest in allowing players to scan physical items into games in the past, but a recent patent suggests that it's going all in.

Fresh off the heels of a showcase at CES 2022 detailing the technical specs of the PS VR2, Sony has updated a patent for even more VR technology. The tech in question would allow players to scan real-world items into virtual reality, making anything interactive in the VR space.

This patent isn't actually anything new as Sony filed it on June 23, 2021, however, the patent office took issue with some of its claims requiring the tech giant to rework some details and resubmit. It would seem that, as of yesterday, Sony and the patent office have begun moving forward with the process following updates and revisions by Sony.

RELATED: Sony Says No Comment on Backward Compatibility for PSVR2

It's currently unclear exactly what the applications of the patent would be for. There are obvious ways that technology like that could be implemented in video games, however, with how much is still unknown about PS VR2 and the titles coming to it, there's not enough information to say for sure that it's going to be used in VR games. The tech could just as easily be used for non-VR experiences as well depending on the game.

It's important to note that Sony hasn't been granted the patent just yet and that it's still being processed so fans shouldn't expect this tech to be featured in games any time soon. It seems to be using similar ideas as the "anything can be a controller, yes, even a banana" patent that the company filed in March, but with a bigger scope in terms of application.

According to the patent mock-up, it seems as if players will be able to scan larger items than the handheld ones featured in the banana patent such as full-sized lamps. The only caveat seems to be that players will need to be able to have a 360-degree view of the item in order to bring it into the digital world.

While this might seem like a dream come true for big VR fans, there are still plenty of hurdles to jump before it can truly be included in a game as a one-to-one recreation. With the growing popularity of the Metaverse, this technology seems like it could be used for VR recreations of real-world spaces, however, Sony hasn't said much about whether or not its newest VR headset will be usable in spaces like that or if the company is even interested in it.

PS VR2 is in development.

MORE: How PlayStation VR2 Specs Compare to the Original Model

Subscribe to our newsletter

The simplest 3D scanner on the example of a smartphone Sony Xperia XZ1 Compact

I share my experience of 3D scanning and printing of busts (souvenirs).
Until now, 3D scanners have been an expensive and inaccessible way to digitize, sometimes it's easier to draw a simple model by hand. But what if you need to create a bust or a model of a real object? This is where various tricks come to the rescue in the form of digitizing from photographs. nine0003 But in modern flagship smartphones, such functions have long been built in. The review will focus on the 3D Creator application for Sony Xperia.

Hello everyone!
Quite by accident I found out that there is a fairly good and inexpensive way for 3D scanning.
Specifically, the 3D Creator application for the Sony Xperia XZ1.

The 3D Creator app is free to download, but without a Sony Xperia smartphone it won't be much use. The program is a 3D model builder that allows you to make a 3D model with relatively low detail and post-processing from a series of shots from a good rear camera. nine0003 I found a relatively good Sony Xperia XZ1 Compact smartphone, which is equipped with a laser sensor suitable for creating 3D models.

Features:
Brand: Sony
Model: Xperia XZ1 Compact
Processor: Qualcomm Snapdragon 835, GPU Adreno 540
NFC: Present
OS: Android 8.0
Protection: IP65/IP68
Display: 4.6" IPS Triluminos Display Corning Gorilla Glass 5
Screen Resolution: 1280x720
Rear camera: 19 MP (Sony G Lens wide-angle optics) EIS stabilization, laser and phase detection autofocus, slo-mo super slow motion 960 fps.
Aperture f/2.0 Sony IMX400 Motion Eye
Front: 8 MP Sony IMX219PQ (Ultra wide angle lens up to 120°)
Battery: 2700mAh supporting QC3.0
Interfaces: BT 5.0, aptX HD, USB Type-C, Wi-Fi a/b/g/n/ac dual-band, 3G, 4G LTE
Additional features:: 3D scanning application

So, the peculiarity of this phone is the built-in laser rangefinder for autofocus, which is adapted for 3D scanning in the proprietary 3D Creator application. Scanning involves an accelerometer, a laser range finder and a rear camera.

There are a number of preset modes (portrait, bust, object scanning), as well as built-in processing (elimination of noise and defects) and post-processing (rotation, cropping, etc.). The result is *.mesh and *.stl for a regular 3D printer. nine0003
The result of the scan is extremely dependent on the lighting, the background (and dynamic objects in the background), the stability and accuracy of the scan itself. The result is not always satisfactory.

Built-in processing reduces noise, reduces detail and heavily blurs objects.

The scanning process is simple - select ~~victim~~ object, select a background (light, monotonous, if a person is on a chair, then in an empty room or against a clean wall). nine0003
Preset modes have their own scanning patterns - focus marks, direction of movement, etc.

As a matter of fact, the scanning process itself is not complicated - just rotate the smartphone along the pattern, trying to maintain the radius, uniformity, etc. The more accurate you do and the smoother the lighting, the better the result. It doesn’t always work out the first time, but there is an opportunity to “rescan”.

The result looks creepy - the application imposes a photo texture on the 3D model. It turns out so still a cabinet of curiosities. nine0003
Initially, the application was positioned as a "hype" for creating animations and 3D collages on Instagram and other social networks. With the ability to order a print in the store. But… the application allows you to export the original *.mesh model, as well as get a ready-made *.stl-file for printing.

When exporting, we get an archive with scan results.
File contents: *.mesh point cloud and textures in jpg.

But there is a separate option to generate a *.stl file that can be loaded into any 3D printer slicer. The screenshot shows the result of a bust scan. The printer prompts him to print 1:1. nine0003
The result of printing on a photopolymer printer

Detailing, of course, so-so. Will go for paint.

Such figures are suitable, for example, as a souvenir caricature, for example, if you paint / decorate a figure, for example, with a wig.

Or “make fun” by painting it creepily for Halloween

Complex or small objects, for example, a bolt thread, the smartphone does not scan - there is too much noise, which is then cut out by the application. As well as it is impossible to make high detailing. nine0003 Smartphone / 3D Creator app is suitable for creating simple firukas, busts, commemorative models with low detail. Often the resulting models require "refinement" - the removal of scanning errors. Although, if you provide conditions and “fill” your hand, then there will be fewer such errors.
Thus, you can get a very simple (and inexpensive) hobby 3D scanner, the scanning itself does not require special skills or software, and the process itself is quite fast. Shooting a bust takes an average of 2-3 minutes of leisurely movement. It can be faster if you get used to it. Printing models on a photopolymer printer took 2 hours. nine0005

As for the application, it did not start on my other smartphones. Editing *.mesh is still a pleasure, as well as editing stl.
Looking ahead, I’ll say that Samsung also has a similar application and scanner, but 3D Creator on Xperia works much better. Perhaps in the future I will compare these functions.

Link to the application and description of 3D Creator on the official website.

How to create realistic 3D models with the Sony Xperia XZ1 Compact / Sudo Null IT News0001

As you already know, one of the highlights of the Sony Xperia XZ1 and Xperia XZ1 Compact smartphones is the Motion Eye camera. But it is interesting not only because of the high-quality shooting of photos and videos, but also because of the innovative 3D Creator function. We have already written about the capabilities of the camera itself, and today we will talk about what can be created in the 3D Creator application using the Sony Xperia XZ1 Compact as an example.

So, the 3D Creator function is available inside the application of the same name. In Russian firmware, it is simply called "3D Model Constructor" and is available on the XZ1 Compact right out of the box. And before you get started, we recommend that you update this app via Google Play as it is constantly being improved. nine0005

3D head and face scan

The application contains four basic modes: face scan, head scan, food scan and freeform scan. The last mode is recommended only for advanced users, so first we will deal with food, head and face.

At the very beginning, it is recommended to scan the face - this mode is used as a calibration, plus in the process you will understand exactly how the scanning takes place. And it happens simply. You simply move the device in the specified direction while the smartphone collects data on the physical movement of the smartphone in space, and the camera "scans" the texture of the face. The 3D model of the face looks specific, like the back half of the head and neck is cut vertically, but this model is used where a 360-degree rotation is not required. For example, when creating three-dimensional animated avatars for further use in augmented reality mode. And also to save 3D stickers that you can save it and send to a friend in any messenger. nine0005

But don't worry, a 360-degree head scan mode is also provided. True, before moving on to it, you will first have to scan the face. The resulting model can be uploaded, for example, to Sketchfab and then viewed on any device.

3D food and freeform scanning

While your friends are posting food photos online, you can take it to the next level and post a 3D model of a cake or a bowl of soup. The modes of scanning food and arbitrary shapes are separated, and in the first case, the smartphone recognizes typical shapes and uses blanks when building 3D models - this greatly simplifies and speeds up the process. nine0005

It is recommended to switch to working with arbitrary forms with a certain skill. But we can immediately give some useful tips. Try to make sure that the subject is evenly lit - if it is under a table lamp, move it to the side to remove hard shadows. And if it takes place on the street, then it is better to move into the shade - the result will be better. And, of course, do not demand the impossible from the application. 3D Creator is unlikely to be able to model in detail every needle on the back of a hedgehog, but with a cactus in a pot, a smartphone will not have any problems. nine0005

Augmented reality, animation and 3D printing

The augmented reality photography mode allows you to use not only pre-made 3D models, but also your own 3D avatars. All this will be available for use when shooting in the standard AR effect application.

Not many features, though, and it's not the only application that takes advantage of 3D scanning capabilities.