The following tutorial gives an introduction to Spell Hyperparameter Searches that builds on the earlier Transfering Style guide. If you haven't already completed that guide and are new to Spell, you might want to try it first. Because the style transfer framework neural-style-tf has many configurable parameters, it is a great use case for exploring the parameter space with Spell's Hyperparameter Search feature.
This tutorial assumes you already have Spell installed, if not, follow the instructions here.
In the below images, we took the style from this Ralph Steadman illustration:
Ralph Steadman Saiga Antelope, 2017 source: Critical Critters
And transfered it onto this image of our teammate's beloved cat:
To get these new images with different parameter values:
Let's start with uploading the images we'd like to use, both as style to extract and as content to transfer style onto. If you already did this in the previous guide, feel free to re-use those images or upload a new set. To stay organized, we can create a directory called
neural-style-imgs containing two sub-directories:
styles. Put in the
neural-style-imgs/style directory any and all "style" images that you want to transfer, and put in the
neural-style-imgs/content directory any and all "content" images that you want to have styles transfered onto.
$ ls neural-style-imgs content styles $ spell upload neural-style-imgs
NOTE: If you already did this in the previous guide, please skip this step and instead re-use the run id from earlier.
In order to run neural-style-tf, you will need the required VGG-19 model weights. This pre-trained model is about 500MB and can be downloaded quickly on a cheap CPU machine using Spell:
$ spell run --machine-type CPU 'wget -O imagenet-vgg-verydeep-19.mat "http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat"'
A truncated response to this run:
✨ Casting spell #1… Run created -- waiting for a CPU machine. ... ✨ Run is running http://www.vlfeat.org/matconvnet/models/beta16/imagenet-vgg-verydeep-19.mat ... Length: 576042600 (549M) [text/plain] Saving to: 'imagenet-vgg-verydeep-19.mat' ... (15.8 MB/s) - 'imagenet-vgg-verydeep-19.mat' saved [576042600/576042600] ... ✨ Run 1 complete
The run identifier for this run (in this example it's
1) is important, so don't forget it! We will be using this identifier to mount the downloaded dataset results into the file system of a new run using the mount flag
Using the Spell Hyperparameter feature with neural_style.py
What's great about
neural-style-tf is that it supports a large list of arguments that can change the behavior of the CNN and produce very different images. We are going to be using
--temporal_weight with different values to see how it affects our output image. The default
1e4, so we will try
1e7 in addition to the default. The default
2e2, so we will try
2e-2 in addition to the default.
We are going to re-use the
spell run command we used in the previous guide, but instead of
spell run we will use
spell hyper grid and in addition to all the previous arguments we will add a couple of
--param arguments for each
neural_style.py argument that we would like to vary across our grid of runs. The
--param arguments will specify a
NAME for the value and a comma separated array of values to use for that parameter, and you must then replace the value for parameter in the actual run command with that name wrapped in
: like so:
:NAME:. This tells
spell hyper grid where to swap the given values into the command for each run in the grid.
Also don't forget to change
runs/1/imagenet-vgg-verydeep-19.mat to your run id from Downloading Datasets above or in the previous guide. If you don't remember your run id, you can find it in your list of previous runs using
spell ps or by checking your runs list on the runs detail page in the web console.
$ spell hyper grid --github-url https://github.com/cysmith/neural-style-tf \ --mount runs/1/imagenet-vgg-verydeep-19.mat \ --mount uploads/neural-style-imgs/styles:styles \ --mount uploads/neural-style-imgs/content:image_input \ --machine-type K80 \ --param STYLE_WEIGHT=1e4,1e1,1e7 \ --param TEMPORAL_WEIGHT=2e2,2e6,2e-2 \ "python neural_style.py \ --style_weight :STYLE_WEIGHT: \ --temporal_weight :TEMPORAL_WEIGHT: \ --style_imgs antelope.jpg \ --content_img cat.jpg"
✨ Casting spell #2…
The transfering will take a bit of time for each run (around 5 minutes on a K80 for 512x512pixels, the default size) and will result in the newly generated image living inside
image_output/result/result.png. To stop viewing live logs, type
Interacting with the Spell Hyperparameter Search web view
To best visualize and interact with your Hyperparameter search you should visit the hyper search details page in the web console. Click on the row that corresponds to this
hyper grid search, and you should see the details of your search, and below a list of all the runs that were launched with the respective parameters that the search used for that specific run. You can click on the run id for each row to view the Run Details Page and scroll to the
Resources section to download the output image for that run to see how the particular parameter values used for that run affected the image output.
What we can see from the grid of images from the output of this particular run, is that
--style_weight has a big impact on the image, obscuring the original content with the style much more as it is raised, but
--temporal_weight doesn't seem to affect our images very much at all!
Play around with the
spell hyper grid feature and the other Hyperparameter search features of Spell and see what you can create!