March 7, 2018

The low down on Spookfish and computer vision

One of the new emerging tech phrases you may have heard of recently is “computer vision” but as usual, most people drop some jargon without actually knowing what it is. For that reason CN&CO sat down with Spookfish AI to get a better understanding of what the field entails. Founded by three university friends, Spookfish is a technology company based in SA that aims to be the leader in leveraging computer vision techniques to build practical products the South African context. To date, they are focusing on three products, namely a digital visitor management system using facial recognition, a cinema audit tool, and a construction site audit tool to enable live notifications of progress delays on site.

First things first though – what is computer vision? The answer is fairly simple according to Stephen Bydawell, one of the founders of Spookfish:

“Computer vision is the process of training artificial intelligence (“AI”) models to interpret what is in an image, through repeatedly training the model on many similar images containing what you want it to learn. In other words using a computer to analyse and interpret data from an image with machine learning.”

The most established computer vision use cases are:

  • Self-driving cars: cameras installed on the cars learn the environment around the car, giving the car the ‘sight’ similar to that of a human when driving
  • The medical field: various computer vision models have been trained on thousands of images (scans, photos or x-rays) of specific diseases, with the resulting models able to predict the type of disease from new images of the infected area, assisting doctors with diagnostics
  • Amazon Go: Relatively new use case, where Amazon installed hundreds of cameras on the ceiling of its physical retail store. The cameras were trained to learn all the items on the shelves, enabling customers to simply pick up all the items they want off the shelves and walk out – the model ringing up all the items and charging them straight to the customer’s Amazon account without the customer ever going to a till.
  • Apple facial recognition: Apple trained their recently released facial recognition model on hundreds of thousands of faces, giving them a high degree of accuracy in identifying whose face belongs to a phone, enabling payments to be done by just looking at your phone’s camera.

It should be noted that computer vision case studies are rarely products by themselves. All of the above examples use various other elements of software to build the platform or user interface that makes the product valuable to customers.

Spookfish focuses on areas not traditionally targeted by computer vision, but where a large amount of visual data can easily be obtained. Overlaying computer vision to a problem often solved manually can increase efficiency, bring insights not previously attainable, and make monitoring feasible where the manual alternative wasn’t an option beforehand.

Some sample screenshots from the Kenai and Spookfish system

A practical application that Spookfish has developed is a visitor management product, Kenai. Most of use can relate to the cumbersome experience of arriving at a client’s office, only to have to fill in some worn logbook with details which are never properly checked, and if there are going to be repeat visits to those particular offices it becomes a painful experience.

“What we have done is replace traditional logbook with a digital sign-in system run off an iPad. To this we added facial recognition to make returning customers sign in by simply having their photo taken. The goal here is to replace a manual, outdated process with a fully automated system, while adding a layer of cutting edge technology to better link the company’s first impression with their customers in the lobby to their overall corporate image” says Bydawell.

The second product aims to tackle the auditing movie seat attendance and cross referencing it to tickets sold. While a niche product it is a laborious and expensive task if done manually and currently Ster Kinekor  does not have a holistic view of how many tickets they have sold vs the number of people who attend movies. As such there is a lost revenue stream which by adding computer vision to do this audit in real time enables management of cinemas to have insight into actual movie attendance vs that predicted from ticket sales. This enables head office management to put in place measures to reduce fraud in cinemas where the model indicates high variation between sold tickets and actual attendance.

The third product and potential for the greatest cost savings by a client concerns the construction industry. Construction sites have been notoriously difficult to manage, with the contractor, bank and developer all having slightly conflicting interests on site when it comes to deadlines and penalty clauses for delays. Having a live feed trained to pick up delays in production, as well as giving live confirmation of weather and other delays on site gives all parties a fair platform to assign remedial action to the party who caused the delay. Given the high cost of each construction project, being able to pick up delays as they happen is crucial to reducing cost overruns and improving alignment between all parties involved in the project.

Computer vision is an industry growing at an incredibly fast pace, and one that isn’t yet widely understood or utilised in South Africa. If you would like to contact Spookfish about any of the information above or alternative uses of computer vision please reach out to info@spookfish.io or one of the CN&CO team.