Computer vision systems on edge devices have enabled us to deliver advancements in machine learning in amazing new ways, and everyone is excited about the future. Using Clarifai’s Mobile SDK, Android and iOS devices are making intelligent decisions remotely, instantly and with minimal computational overhead.
But with advancements in mobile AI come many new challenges, not the least of which is the fact that you will want your app to be making critical decisions in situations with intermittent, or even non-existent internet connections. Fortunately, there are well-established techniques that can help ensure that your system is making effective choices, even when it is offline.
Here are 4 secrets to making sure that your application is performing optimally on edge devices.
1: Give it Value and Weight
You want your application to connect with your business in an intentional way, and you will want to be able to quantify success. Often this quantity is represented with good old fashioned dollars and cents.
Provide your system with some kind of nominalized value that represents the benefit of a helpful outputs, and the cost of an unhelpful ones. For example, you may decide that highly helpful matches are worth $0.12, moderately helpful matches are worth $0.03, no response are worth $0.00, and unhelpful matches are worth -$0.13.
2: Validate
Next, you will want to test a large “validation” data set - one where you already know the desired outputs for each item in the set. Ideally, this will be different data than what you have used to train your system. Testing your system against validation data helps you to understand the behavior of your system out in the wild.
Record correct outputs, incorrect outputs, and null responses. You will also want to take note of the associated probability for each response.
3: Run the Numbers
Once you have the results of your test, you can simply compute the net reward for the system given different probability thresholds. Roughly speaking, your calculation might look something like this:
For each probability interval that you would like to consider:
(Percentage of highly helpful matches) x ($0.12) +
(Percentage of moderately helpful matches) x ($0.03) +
(Percentage with no response) x ($0.00) +
(Percentage of unhelpful matches) x (-$0.13)
4: Pick the Winner
The final step is simply to select the threshold level with the greatest total reward.
Cutting Edge AI, Unbeatable User Experience
By normalizing your outcomes using probability scores, and weighting the results with some kind of numerical value, you can ensure the best possible user experience - no matter where your users are.
We understand that assigning numerical rewards to outcomes can be difficult, and often data is not available to make these assignments in an informed way. Clarifai’s large pre-optimized data sets can help your app get to work, even if you do not have any user data, or the user data you have does not provide information about the consequences of the system behavior.
Clarifai can help you understand how to structure your mobile development efforts for beneficial advancements over time, and can even ensure that your system responds in predictable ways as you add new data or make other changes to your model.
