Three ways data science is unbaking the cake

Here are three ways data science is already doing the impossible.

1. You can’t un-blur an image

While it’s easy to digitally blur a sharp image, you can’t sharpen data that isn’t there or correct motion blur that’s baked into a RAW image file. You can manually amplify or reduce that data, to some extent, in a program like Photoshop but your options are limited and the results are rarely worth the time it takes to achieve a noticeable difference.

In photography, a dud image is a dud.

However, deep learning provides an algorithmic alternative that can replicate existing data and use it to replace information that’s not desirable. In layman’s terms, this means you can un-blur an image, as illustrated in this article written by Ceshine Lee on Medium.

You’re not going to get pin-sharp images out of this technology but it can make information in low-quality smartphone images visible. For example, blurred number plates or faces submitted to the police as evidence by the public.

2. You can’t extract the vocals from a song

If you’ve ever used voice type on your phone, you’ll understand how difficult it still is for machines to accurately interpret and transcribe human speech accurately. Now, try doing this in a club with music pumping out over the sound system and you’re getting closer to the challenge of extracting lyrics from a song.

Before lyrics can be extracted, algorithms need to be able to distinguish lyrics from other noise signals. Then, the algorithm needs a way of separating the multiple vocal and backing tracks that are typically layered over each other in so much of today’s music.

Difficult stuff.

However, it can be done by using convolutional neural networks (CNNs), as explained in this in-depth article written by Ale Koretzky, which describes how deep learning is capable of recognising the patterns in human speech. Likewise, deep learning is also capable of recognising the structure of music, which means separation can be achieved.

Now, separation is only the first stage of the process but it’s the key step that makes everything else possible. Extracted vocal data may be incomplete or include small amounts of music data that could result in errors. The solution is actually very similar to unblurring images as a means of filling in missing data.

If you’re interested in the specifics of the entire project, be sure to read Ale’s article.

3. You can’t un-crop a photograph

Another common photography problem is sizing images for various displays. Simply enlarging an image will degrade quality, resulting in unsharp images or even pixelation on large displays. Once again, reducing the size of the image is relatively simple but creating image data that doesn’t exist is far more challenging.

In fact, most photographers would have considered it impossible until recent developments in computational photography.

Photo Uncrop by abhinavs95 offers one solution to this problem, algorithmically extrapolating cropped images by using the data within the existing image to digitally un-crop the photograph.

“The goal of this project is to build a model that extrapolates cropped images of outdoor scenes. We use an encoder-decoder architecture to capture contextual information from input cropped images and generate an extrapolated image based on the context. We also use a Generative Adversarial Network in conjunction with the encoder-decoder framework to obtain highly detailed results. Finally, we use a post-processing technique called seamless cloning on the generated output to further improve the quality of the results.”

The algorithm is capable of recreating the shape of mountains and details like pathways and shadows with an impressive amount of accuracy.

How close are we to unbaking cakes?

Well, even data science can’t unbake a cake (not yet, anyway). Let’s be honest though, cakes are made to be eaten, and it would be far more delicious if deep learning was able to knock up a few baked goods instead. Like these “cupcAIkes” created by Tiny Giant, Sharpshooter Digital and Little House Kitchen.

Some things are best left not undone.

Henry Carless, PPC and data scientist at Vertical Leap.