An ML Newsletter from Novetta
Welcome to the December 2020 BASELINE, Novetta’s Machine Learning Newsletter, where we share thoughts on important advances in machine learning technologies. This month we cover the following topics:
- A major breakthrough demonstrating the real world impact of recent deep learning advances with AlphaFold2
- A new state-of-the-art object detection model in Scaled-YOLOv4
- Tiny machine learning on IoT devices with MCUNet
Solving the Protein Folding Problem with AlphaFold 2
Predicting the 3D structure of a protein using its amino acid sequence has been an open problem in biology for the last 50 years. DeepMind’s AlphaFold 2 achieved a major breakthrough on this front at the CASP competition. DeepMind is well known for creating AI to beat the world’s best Go and Starcraft players, with the eventual goal of addressing applications that make a real-world difference. Until now the primary way to determine a molecule’s structure was through high-tech x-ray technology, which can be extremely time consuming and expensive. AlphaFold 2 generated almost perfect predictions in two thirds of test cases and was highly accurate for most predictions in the remaining third. This sort of breakthrough, with the potential to get accurate predictions at much faster speeds, will allow biologists to ask new questions. This breakthrough demonstrates how machine learning methods, which automate tedious tasks, can serve as “force multipliers” for researchers and analysts. In this case, it will help researchers and scientists develop a deeper understanding of diseases and human genomes, potentially leading to new drugs to combat diseases.
A New Leader in the Race for State-of-the-Art Object Detection
The quick releases of YOLOv4, YOLOv5, EfficientDet, and PP-YOLO this past year have set new bars for object detection models. The release of Scaled-YOLOv4 represents a new undisputed state-of-the-art model in terms of both speed and accuracy. It has the best results in terms of speed-to-accuracy ratio for the entire range of 15 FPS to 1774 FPS on the COCO baseline dataset. As with YOLOv5, Scaled-YOLOv4 was written in PyTorch, whose familiarity to most machine learning practitioners makes it extremely easy to use. PyTorch also helps to speed up training times. Users of object detection models no longer need to partake in a nuanced side-by-side comparison about which model is best, the answer is clear (for now).
A New Frontier in Tiny Machine Learning
As GPU and TPU processing becomes more accessible, deep learning models have grown in size and memory usage to approach problems previously considered computationally infeasible. Increased attention is being applied to the opposite frontier of machine learning: tiny machine learning. This subfield aims to make models small, with low memory usage, so they can be deployed to IoT devices. Researchers at MIT have released MCUNet, an efficient framework whose inference methods have less memory usage and increased speeds compared to TensorFlow Lite Micro and CMSIS-NN. MCUNet achieves the highest Top1 performance on ImageNet on a commercial off-the-shelf microcontroller. It also achieves state-of-the-art accuracy on visual and audio wake word tasks. The authors discuss the broader impact of MCUNet as lowering the bar for those who can access the benefits of machine learning, since models can be applied to $5 off-the-shelf devices. This could mean that areas of the world with limited internet connection can still use ML for some tasks. MCUNet opens up new opportunities in microcontroller-based model deployment in healthcare, retail, and agriculture applications.
This research was performed under the Novetta Machine Learning Center of Excellence.