Train a Custom Visual Recognition Model Using Clarifai's Python Client

In December, our v2 API graduated from developer preview to general availability. Don't they grow up so fast?! As such, here is a useful tutorial on how to use one of Clarifai v2 API's most popular features - Custom Training. Custom Training gives you the ability to teach new concepts for our technology to recognize using only a handful of examples and a couple lines of code!

If you've seen our new v2 API documentation, you've probably noticed that things are a bit different than they were in v1 - new SDKs, a cool new visual tool, and new projects waiting for you to build them using Custom Training! In one of our previous blog posts, we talk about how to use the Preview UI for Custom Training. While that was awesome (like really awesome), we're gonna crank it up a gear and show you how to do some of the same things using PROGRAMMING.

It isn't as flashy as pulling up our sleek, chic UI, but it is just as rad. You can go to your friends and show them your mad teaching-a-computer-how-to-see skills!

---

An API Client

First thing's first, we are gonna need to find a way to interface with the API with our code. We have written a few different clients for you to use in several different languages. For this tutorial, we are gonna stick with the Python client. Who doesn't like programming languages that came out of Monty Python?

NOTE: Be sure to have installed a Python that is at least 2.7 or higher. If you aren't sure what which one you might have, go into your terminal and check out what python --version gets out for you. For instance this is what happens on my system:

$ python --version
Python 2.7.10
# Or if you are using Python3
# Python 3.5.1

Also to install the Clarifai API Client, we are gonna make sure we have pip, the Python Package manager:

$ pip --version
pip 9.0.1 from /path/to/Python/2.7/site-packages (python 2.7)
# or if you are using Python3
# pip 9.0.1 from /path/to/python3.5/site-packages (python 3.5)

With those two out of the way, we can now finally get to installing the Python Client!

$ pip install clarifai
# NOTE: If this doesn't work, consider having a sudo in front of the command 
# to tell your computer who is boss.

If that went off without a hitch, we need to do one more thing before we can write our Custom Training. We are going to add our Client ID and Client Secret into our app. Every time you want to hit Clarifai, you want to be sure to have these two at least! No matter what language you are using, this will be the only way you can get access. So head over to the Developer website, and create yourself a new application:

Once that is done, head back to terminal and type in:

$ clarifai config

When you hit enter, it will prompt you to put in your Client ID and Client Secret, respectively. That way, when we make calls, this doesn't have to be saved in your project but on your file system so long as you are using that particular Application:

CLARIFAI_APP_ID: []: ************************************YQEd
CLARIFAI_APP_SECRET: []: ************************************gCqT

Congratulations! If you made it this far, you have successfully added the Clarifai Python client to your system. Go you! Now, with that out of the way, we are gonna write some code!

Add Images with Concepts

With Custom Training, you can teach Clarifai to recognize whatever you want in an image! So for me, I am gonna teach Clarifai about corgis (because corgis are the best). For this part, you can find 10-20 images that you want to train your model with. This will be known as the “training set”, a set of images you use to train with that will have positive and negative examples of whatever concept you are trying to get Clarifai to recognize.

So, I gathered 15 images of corgis shot from different angles (including the all-important "corgi butt" pose) with various background objects to make sure Clarifai can recognize what I want it to. Though all my images are JPGs, Clarifai supports JPG, PNG, BMP, and TIFF files! Depending on what you want to train for, it can be better to have a more realistic photo of what you want Clarifai to be looking for and sometimes it is okay to want just the object itself in front of a solid background.

In my project folder, I made a folder to house all of the images I collect for a concept. So all my images in my corgi/ directory will house images that will have ‘corgi' as a concept.

To add an image object with the concept of corgi, we are gonna write:

from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage

# Our app
app = ClarifaiApp()

# Adding our image onto our app
app.inputs.create_image_from_filename('./corgi/corgi-1.jpg', concepts=['corgi'])

When we run this (python.py), we can see our input is added onto our app on the Preview UI:

To utilize the power of programming, we are gonna automate the process. Let's make a function that will import all of our images inside a directory with the same concept. In this case, we will have all of the photos under corgi/ with our ‘corgi' concept!

# Packages for reading local files
import os
from glob import glob

def create_image_set(img_path, concepts):
    images = []
    for file_path in glob(os.path.join(img_path, '*.jpg')):
        print(file_path)
        img = ClImage(filename=file_path, concepts=concepts)
        images.append(img)

    return images

This will allow us to easily take in any sort of images, no matter where, and also allow for us to set some images to have concepts! This is just some of my code and you are more than welcome to improve it to fit your needs.

NOTE: For this function, we are making the assumption that our number of images per batch is relatively small. If you are sending large sets of images at one time, with numbers around 100,000+ images per function call, you might want to consider abstracting this function to asynchronously handle sending out the request.

So now our code will look more like this:

from clarifai.rest import ClarifaiApp
from clarifai.rest import Image as ClImage
from glob import glob
import os

def main():
    app = ClarifaiApp()
    image_set = create_image_set('corgi/', ['corgi'])
    app.inputs.bulk_create_images(image_set)

def create_image_set(img_path, concepts):
    images = []
    for file_path in glob(os.path.join(img_path, '*.jpg')):
        img = ClImage(filename=file_path, concepts=concepts)
        images.append(img)

    return images

if __name__ == '__main__':
    main()

Notice we call our create_image_set in the main method and set our image_set equal to the returned results. Then we send this into app.inputs.bulk_create_images. Now you should be seeing all of the images you have added when you run the script!

Now we have added some images with our corgi concept! Let's try to add this onto a model and train!

Create a Model and Train!

Since we have already added our inputs, all we need to do now is create a model and train it! To avoid adding duplicate inputs, we are going to comment out the image stuff and change our main function to only:

def main():
    app = ClarifaiApp()
    # image_set = create_image_set('corgi/', ['corgi'])
    # app.inputs.bulk_create_images(image_set)

    model = app.models.create(model_id="puppy", concepts=['corgi'])
    model.train()

Here we created our model, with a model_id of “puppy” and concepts that are on the model align with concepts that we have on our model. From there, we make sure to call train. Voila! We have trained our model to recognize the concept of corgis!

Now let's say we wanted to use our model and see whether or not it works, we can change our main to be:

def main():
    app = ClarifaiApp()
    # image_set = create_image_set('corgi/', ['corgi'])
    # app.inputs.bulk_create_images(image_set)
    # model = app.models.create(model_id="puppy", concepts=['corgi'])
    # model.train()
    model = app.models.get('puppy')

    url = raw_input("URL of image: ")
    # if on Python3, change the above line to
    # url = input("URL of image: ")

    print model.predict_by_url(url)['outputs'][0]['data']['concepts']
    # if on Python3, change the above line to
    # print(model.predict_by_url(url)['outputs'][0]['data']['concepts'])

We will ask our user to give us a URL and then we will print against our puppy model! This will give us the output of all of our concepts on our model:

$ python app.py
URL of image: https://s-media-cache-ak0.pinimg.com/236x/37/5d/48/375d488c51f51c72a01e0e88420becca.jpg
[{u'app_id': u'b066d570bcae40b3b0d5d50176d09cef', u'id': u'corgi', u'value': 0.9865137, u'name': u'corgi'}]

That's it for this tutorial! There are a number more functions and ways of doing things in Custom Training. If you ever need any help, be sure to reach out to us at [e-mail]. We are more than happy to help!

Before I send you on your way, here are some simple tips and tricks to remember about Custom Training:

1. Be sure to have your inputs (images) on your application, and tell them what concepts (or not-concepts) they are associated with!
2. Make sure you have a model (a place to put a group of concepts) and concepts! Just having images on your application isn't enough.
3. If you are unsure how to access the API using any of the clients, be sure to check out the repo's documentation. For Python, you can see all the available functions here.

Go out my compatriots of the world. Teach computers how to see the world from your perspective!

Comments

[Elizabeth]

this article is awesome!

[Prince Wilson]

Glad you like it!!