Machine Learning for Product Development
(This article also appears on LinkedIn)
Yeah, I get it. The title sounds attention- and click-grabbing. And I’ll be honest with you: this is not the ultimate machine learning algorithms-powered supply-chain solution for your multimillion dollar manufacturing company. In fact, you are absolutely right to be skeptical about anyone claiming they’re „using AI“ and are „applying deep learning“.
But the question remains: how do we combine human and machine intelligence for product development? And how do we do we apply it in practical machine learning applications? If you’re interested in one of my little machine learning examples, read on! And if you want to buy the finished products, click here and here! :)
The power of DCGANs
The Udacity AI Nanodegree briefly visits neural networks known as „Deep Convolutional General Adversarial Networks“, or DCGANs in short. I’m not going to bore you out with lots of theory here, but let’s look at this machine learning algorithm for a minute! And if you want to know more, there is this excellent article by Rowel Atienza.
GANs (the main part of the above mentioned algorithm) were introduced by Ian Goodfellow in 2014. The typical explanatory example for this kind of neural network is that of a counterfeiter (called the Generator) and the police (called the Discriminator). The counterfeiter tries to come up with more realistic looking fake bills and the police gets better at detecting which ones are real and which ones are not. They race themselves until you have one network that is very good at generating fake images and another that is very good at detecting them. You then take whatever network you need for your purpose.
Earlier in 2018, NVIDIA used this method to generate high-quality realistic looking images of non-existing celebrities. Check out their YouTube video to see what else they can do: https://youtu.be/G06dEcZ-QTg
Not real people. Yeah, I’m as bamboozled as you are. If you look closely at her eyebrows though, you can see where the software got it wrong. (Image: NVIDIA Corporation 2018)
Imagine it like this: This resulting neural network tries to generate images that look like the ones it was trained on. In the above example, it learned that the training pictures consist of eyes, noses, cheeks, hair and all of that in a specific combination. So the resulting images are not an exact copy of a training picture, but something new, that looks like a training picture. If you train it on pictures of cars, you get a picture that has something with wheels, doors, windows, etc. on it.
The technology is quite promising, but very difficult to apply. As all neural networks, it needs a lot of learning material and the hardware requirements are quite high (I guess if you’re NVIDA though, that doesn’t matter too much). On top of that, the hyperparameters of the networks (learning rate, etc.) are very sensible to adjust. Shift them even a tiny fraction to the wrong side, and you’ll end up with nonsense.
Think of it as a very interesting, but incredibly difficult to care for fish for your aquarium. Yes. Pretty for a while, but it will frustrate you over and over again. Fun for the whole family! (Image: nokkaew / Shutterstock)
Naturally, I wanted to implement this myself!
How to sell things made by a machine
If you take a quick look at sites like Shutterstock, you’ll notice immediately that there are a lot of identically looking pictures.
Shaking hands is very important in our society. (Image: Screenshot google.com Image Search)
Especially, when certain big local (or even global) events take place (think Christmas, Chinese New Year, etc.), the pictures tend to look somewhat alike. Many are drawn, e.g. a red background with a symbol of the new Chinese year on it. A question arose: could I capitalize on that? The technology at hand could - in principle - generate countless images in a certain style, once it’s been trained. And it could be trained on basically all images of a certain style that look somewhat identical.
My idea of a digital supply chain looked like this:
Set up a script that detects trending topics and hashtags on platforms like Instagram, Twitter and of course sites like Shutterstock
Acquire training images for a certain trending hashtag from Flickr (because that’s one of the last sites that still offers easy API access)
Feed a DCGAN with these images and train it to produces images that fit the hashtags
Receive a neural network that can generate pictures for the hashtag
Generate hundreds of images and put them on Shutterstock
Hope to sell at least a couple of them so it covers the AWS costs (where I train my neural networks)
So basically, I hoped that once a new event rolled around, newspapers, journals, Instagrammers and whoever else needed something to illustrate content would buy my generated stuff. My system would be considerably faster than anyone who needed to draw up stuff first and once trained, it would create content at virtually zero costs. If out hundreds of easily generated pictures (placed on several stockimage platforms), I would only sell a couple, this should cover the costs of training the network. And since this is automated, with only a few control points (to avoid this), I could just go on sitting in my backyard and check my bank account every few days.
You know where this is going. (Image: AMC sceencap, Breaking Bad)
To be honest, I’m not the first one to think of this. Adam Geitgey wrote about it back in 2017. But while he only played with the idea of commercializing this, I thought „why not“?
Data, data, data
The first thing you notice, after copy-pasting the relevant source code from some obscure Github repository, is that you have zero data to train this with. Yes, there is Flickr, but it seems to be somewhat deserted of newer pictures. Especially if you look for images that are drawn or vectorized (so something that would be useful as a StockImage). It is moments like this that you realize, what a big advantage companies like Google, Instagram and others have when it comes to data.
Of course there are public repositories of images (MNIST, COCO, CIFAR and everything on Kaggle - take a look here and here). But I needed images based on currently relevant events, not just „anything that is available“. Sites like Instagram make it increasinglyinconvenient to bulk-download images (and with good reasons). It is at this point important to remember, that you need a couple of thousand images for this technology to work - or even tens of thousands. There are solutions like the InstaLooter, but how reliable and legal all of this it, I don’t know. My plan was kind of coming apart.
But then I thought I should maybe just focus on getting the technology to work, before throwing down the towel because of missing data. It was during my vacation in Vietnam that I realized most beaches look the same and decided to roll with that. One Flickr bulk-download later and I was ready to go.
Seriously, why do we keep taking pictures of this? (Image: Screenshot google.com Image Search)
Supersize me!
My implementation was based on the one I did during my Udacity course. What I failed to remember was, that the software produces images in a 32x32 size. Here are the first results after some hours of training:
Quite beach-y, right? Still far away from a potential Stockimage. Trying to increase the image size quite quickly brought me to the limits of my Amazon AWS machine - or more accurately, the limits of my bank account.
I’ll spare you the tech details and jump straight to the results instead. Here’s some of the best results I got out of my little creative AI:
I have to admit, I’m still amazed that recognizable shapes start to appear. Apparently, the system figured out that beaches consist of water, sand, sun and more. I’m sure, if the input data was better (read: cleaner), I had more of it and more GPU power, I could push this to NVIDIA-like quality. Still, this isn’t my main job and I seemed to have hit a dead end.
Humans to the rescue
A couple of months later, I was at another beach and I kept thinking about this. I had recently listened to a podcast where someone sold t-shirts over Facebook. And this is where it hit me: Why not combine artificial and human intelligence? I went to Upwork and looked for designers and other creatives:
To my surprise, 20 people almost instantly applied to this. I talked to some of them and then settled for a couple. Each one would receive a different picture, generated from my AI and was asked to return a high-resolution vector image. Without further redo, here are some results:
My idea here was, that it would be a lot easier for a designer to start from a certain picture. If I instead asked for a „picture from a beach“, it might have taken longer and maybe have been more expensive.
Now I only needed to print them on shirts and sell them. Teespring is a very good online solution, that takes care of basically the whole value chain, including printing, shipping, payment.
One Facebook ad campaign later and my digital supply chain is complete!
You can buy my stylish beach bag here and the above shirt here!
Will this bring me the amount of cash needed to quit my job? Certainly not. In fact, it is probably easier to just buy an existing stock image and put it on a shirt to sell. But I liked the idea of how to let artificial and human intelligence work together. Basically, I had people interpret images dreamed up by an AI - awesome thought, isn’t it? The AI did not replace the human part, but the human part got a lot easier: this is what Daugherty and Wilson were talking about. I also liked the idea of how I was able to construct a completely digital supply chain that ships a physical product, without much effort. So bring on the robots! There seems to be much fun ahead! :)
