There’s a common saying: 'you can’t unbake a cake'. It’s a metaphor that illustrates how some things can’t be undone and a pretty good life lesson besides. However, there’s also genuine science behind this expression, referencing the irreversible chemical reactions that take place in the heat of an oven.
Others might prefer the saying 'nothing is impossible' and this is the kind of mindset data scientists need to have. Technologies like AI and deep learning are making the previously impossible possible - so try telling a data scientist that you can’t do anything.
Here are three ways data science is already doing the impossible.
1. You can’t un-blur an image
While it’s easy to digitally blur a sharp image, you can’t sharpen data that isn’t there or correct motion blur that’s baked into a RAW image file. You can manually amplify or reduce that data, to some extent, in a program like Photoshop but your options are limited and the results are rarely worth the time it takes to achieve a noticeable difference.
In photography, a dud image is a dud.
However, deep learning provides an algorithmic alternative that can replicate existing data and use it to replace information that’s not desirable. In layman’s terms, this means you can un-blur an image, as illustrated in this article written by Ceshine Lee on Medium.
You’re not going to get pin-sharp images out of this technology but it can make information in low-quality smartphone images visible. For example, blurred number plates or faces submitted to the police as evidence by the public.
2. You can’t extract the vocals from a song
If you’ve ever used voice type on your phone, you’ll understand how difficult it still is for machines to accurately interpret and transcribe human speech accurately. Now, try doing this in a club with music pumping out over the sound system and you’re getting closer to the challenge of extracting lyrics from a song.
Before lyrics can be extracted, algorithms need to be able to distinguish lyrics from other noise signals. Then, the algorithm needs a way of separating the multiple vocal and backing tracks that are typically layered over each other in so much of today’s music.
However, it can be done by using convolutional neural networks (CNNs), as explained in this in-depth article written by Ale Koretzky, which describes how deep learning is capable of recognising the patterns in human speech. Likewise, deep learning is also capable of recognising the structure of music, which means separation can be achieved.
Now, separation is only the first stage of the process but it’s the key step that makes everything else possible. Extracted vocal data may be incomplete or include small amounts of music data that could result in errors. The solution is actually very similar to unblurring images as a means of filling in missing data.
If you’re interested in the specifics of the entire project, be sure to read Ale’s article.
3. You can’t un-crop a photograph
Another common photography problem is sizing images for various displays. Simply enlarging an image will degrade quality, resulting in unsharp images or even pixelation on large displays. Once again, reducing the size of the image is relatively simple but creating image data that doesn’t exist is far more challenging.
In fact, most photographers would have considered it impossible until recent developments in computational photography.
“The goal of this project is to build a model that extrapolates cropped images of outdoor scenes. We use an encoder-decoder architecture to capture contextual information from input cropped images and generate an extrapolated image based on the context. We also use a Generative Adversarial Network in conjunction with the encoder-decoder framework to obtain highly detailed results. Finally, we use a post-processing technique called seamless cloning on the generated output to further improve the quality of the results.”
The algorithm is capable of recreating the shape of mountains and details like pathways and shadows with an impressive amount of accuracy.
How close are we to unbaking cakes?
Well, even data science can’t unbake a cake (not yet, anyway). Let’s be honest though, cakes are made to be eaten, and it would be far more delicious if deep learning was able to knock up a few baked goods instead. Like these “cupcAIkes” created by Tiny Giant, Sharpshooter Digital and Little House Kitchen.
Some things are best left not undone.
Henry Carless, PPC and data scientist at Vertical Leap.