There are two certainties these days: 1) machine learning is creating an ever more powerful set of tools to accomplish a myriad of complex tasks, and 2) getting out of bed is hard. Now, with some Java and the help of the Clarifai API, you can make an app that helps you resolve the latter with the former: an alarm clock that only turns off once you take a photo of a pre-specified object. For example, here it turns off when it sees a chair:
Special thanks to the @RedRussianBear Mikhail Khrenov for his work on this project.
Prepare Your Clarifai User Account
Clarifai provides deep learning driven computer vision and image recognition service. Your first step should be to sign up for a free Clarifai account.
After you’ve made an account you will need to create an app with Clarifai through the developer interface:
Then, select your application and click “Create API Key.” This will be the ID that your app will use to gain access to Clarifai.
You’ll want to note this API key to use later. Keep it safe!
Create Your Android Alarm Clock App
We will be developing the rest of the app in Android Studio, the official IDE for Android. If you haven’t already, download and set that up. This tutorial will take you through one section of the app at a time, but you can download all the code on GitHub.
We will make a new blank Android project which should include an initial blank activity called “MainActivity.” After your project has been made and the initial Gradle build has finished, we are going to do some additional setup.
First, add the line...
api 'com.clarifai.clarifai-api2:core:2.3.0'
...to the end of your app level build.gradle. This will tell Gradle, the build system, to include Clarifai and give us access to Clarifai’s libraries.
Next, we are going to create a new file under res/xml/ called file_paths.xml, with the contents shown above. We will need this later to store the photos our user’s camera will take.
Finally, we are going to make two additions to our app’s manifest: first, inside the <application> block, we will add a <provider> as shown above to give us access to the photos we take.
Then, outside in the manifest block, we will add the three <uses-*> statements. The first tells Android that our app needs a camera (in order to take photos), the second tells Android that on certain SDK versions our app needs permission to access files (in order to store them), and the last one tells Android that our apps needs the internet (to talk to Clarifai).
Now, to the app itself. Alongside MainActivity, we will add another activity called AlarmActivity. To do this, in Android Studio right click app/java, select new->Activity->Blank Activity. While MainActivity will be the user interface for first launching the app and setting an alarm, AlarmActivity will be what the user sees when the alarm goes off and they need to deactivate it.
Create Your App Layouts
We will first design the layouts and inputs available in each Activity. The blueprint mode of res/layouts/activity_main.xml should look something like this:
From top to bottom, the input elements are a Plain Text called ‘object’, a Date called ‘date’, two Number inputs named ‘hour’ and ‘minute’ respectively, and Button called ‘button’. While you may name them as you wish, you will need to alter some later code to match. The inputs on this view will let our user input what object they want to look for, when the alarm should go off, and trigger the function that sets the alarm up.
Meanwhile, activity_alarm.xml should look like:
We have the TextView on top named infoText, and the Button in the center named photoButton. We will use the former to notify the user of the state of Clarifai processing, and the latter to pull up the camera.
If you don’t want to mess around with the layout editor, then simply copy the contents of activity_main.xml and activity_alarm.xml into your project files.
Our res/values/strings.xml file should look as follows to set the default values for all of our UI elements:
Connect the UI to the Code
Now that we’ve constructed the user interface for the app, we can get to the handler code. Before we begin, include the following imports at the top of the respective files:
We will first address MainActivity. In this activity, the user will decide on a time and an object-to-detect as parameters to set the alarm. To handle this, we will add a new method setAlarm():
First, we retrieve all of our view inputs by id. We then compile the values of date, hour, and minute into a single string, which we parse to create a Date object that the alarm will reference.
Next, we create an Intent to open AlarmActivity. In the Extra for that Intent we will put the contents of object, the name of the object our user needs to photograph in order to disable an active alarm.
Finally, we retrieve the AlarmManager, put the pieces together to set the alarm, and gracefully exit the activity with the finish() function.
So far, we have an app in which we can set an alarm to open another activity of the app. The actual alarm clock elements come in AlarmActivity: playing a sound, and capturing a photo to be processed by Clarifai.
When the Activity is created (when the alarm goes off) we will need to supply what object should deactivate it. We will retrieve that parameter using the Intent extra we set before in the onCreate() function.
Next, we’ll work on playing a sound. In our onStart() method, we set up a MediaPlayer object. We configure it to work asynchronously to prevent the app from stalling out. While our MediaPlayer can play from an arbitrary URI (or even stream from a URL), for simplicity we will retrieve the URI of our user’s default ringtone and play it continuously. With this setup, we send a message to the user via infoText: “Snap a [object]!”.
Now that we’ve gotten our user’s attention, we need to give them a way to take a photo. We do this through takePicture(), which we bind to the on_click field of photoButton in activity_alarm.xml. Inside this method we create an Intent to run a camera app, create a file where the app can place the photo it takes, and store the path to the photo in a member field.
Now we have an alarm clock that lets us take a photo, but to turn off the alarm we need to make sure the photo actually includes the real-world object we photographed when setting the alarm. This is where Clarifai comes in!
Interpret Your Photo with Clarifai
This is the surprisingly easy part: running object recognition on your image using Clarifai. In order to do this, we will make the private class ClarifaiTask, an extension of AsyncTask. We need to run our communication with Clarifai asynchronously as we need to make a network call, which can take several seconds. By default, Android wouldn’t even let our code compile if we put the Clarifai call on the main thread.
Inside ClarifaiTask, we need to implement two methods: doInBackground() and onPostExecute(). In doInBackground() we will connect to Clarifai through a ClarifaiClient object (this is where your API key from earlier comes in — be sure to replace YOUR_API_TOKEN with the one from your own Clarifai dashboard!). Then, we send our photo to the Clarifai ‘general’ model, which recognizes the broadest range of objects.
We use the image predictions sent back by Clarifai to see if the results contain the name of the object we specified when creating the alarm. In onPostExecute(), we check if the object we were looking for was detected, then delete the image since we no longer need it. If we did detect the correct object, we exit the activity (turning off the alarm ringtone). If not, we prompt the user to try again because the object was not detected.
ClarifaiTask is started in our implementation of the onResume() method. When coming back into the app, we check if our photo points to a file. If so, we have just come back from the camera app and should send the image off to Clarifai to check if we should turn the alarm off.
Finally, we need to ensure that when our AlarmActivity is closed, the alarm turns off.
Whew, now the ringing can finally stop... and we did it! We have a visually aware alarm clock running easily on our Android device that we can set to look for any object we think will require us to get out of bed to photograph.
Next Steps with Image Recognition
This is your starter implementation of a visual alarm clock. From here you can extend it to allow repeating alarms, arbitrary alarm sounds, and custom objects-to-detect that you’ve trained Clarifai to look for using Custom Training. For example, you could train Clarifai to recognize your personal exercise shoes or certain landmarks. Image recognition powers are at your fingertips, and you’ve already got your own Android app. What are you waiting for?
