IDG.Net Core
It has been said that converting a software project to open source is either the best thing or the worst thing you could do for it. In the case of Microsoft’s .Net Framework, open sourcing a subset of the functionality as .Net Core has been a resounding net positive, resulting in a lighter-weight runtime with an open development process, a cross-platform-first philosophy, and a compatibility bridge back to the main .Net Framework for apps that need it.
Version 2.1, released in May 2018, rolled in a great many features that complemented this larger plan. Biggest among them: .Net Core Global Tools, a new system for deploying and extending the command-line tools used to manage .Net apps; the Windows Compatibility Pack, providing access to the 20,000 APIs used in the big-brother .Net Framework for Windows-native apps; API analysis tools to identify Windows API dependencies when porting a Windows app; and mechanisms for publishing self-contained .Net Core apps with the latest runtime bundled in.
—Serdar Yegulalp
IDGLLVM
At first glance, LLVM might seem an esoteric choice for our award. A toolkit for building programming language compilers? But the power of the LLVM compiler framework sits at the heart of a number of A-list projects: Clang, Rust, Swift, Julia, and many other innovative projects moving programming languages forward.
LLVM gives developers the means to generate machine-native code programmatically—and without their having to know the vicissitudes of each architecture and platform they want to target. The obvious use case is language compilers, but LLVM also makes possible a whole variety of other applications. For instance, PostgreSQL uses LLVM to dynamically generate code that accelerates SQL queries. Similarly, the Numba project uses LLVM to transform slow Python code into fast assembly language for high-speed number-crunching applications.
LLVM’s two major releases of 2018 introduced a slew of improvements: better support for newer Intel processors, multiprocessor scenarios, mitigations for common processor flaws, tooling for evaluating the performance of generated code on specific CPU architectures, and further work on the LLVM linker, LLD, which can produce standalone executables from LLVM across multiple platforms.
—Serdar Yegulalp
IDGTensorFlow
TensorFlow is an open source software library for numerical computation using dataflow graphs. The graph nodes represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture enables you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes TensorBoard, a data visualization toolkit.
At Version 1.12, TensorFlow is by far the most widely used and widely cited deep learning framework available. While it still supports its original low-level API, the favored high-level API is now tf.keras, an implementation of the Keras API standard for TensorFlow that includes TensorFlow-specific enhancements.
While TensorFlow still supports building dataflow graphs and running them later in sessions, it also now fully supports eager execution mode, an imperative, define-by-run interface. Eager execution mode supports automatic differentiation via the tf.GradientTape API. One of the enhancements in tf.keras is support for eager execution.
Both the Keras API and eager execution mode will be featured in TensorFlow 2.0. While some other APIs will be deprecated in version 2.0, there will be a conversion tool for existing code, in addition to a compatibility library.
Estimators are TensorFlow’s most scalable and production-oriented model type. You may either use the pre-made Estimators that Google provides or write your own custom Estimators. Estimators are themselves built on tf.keras.layers, which simplifies customization. It is usually much easier to create models with Estimators than with the low-level TensorFlow APIs. Pre-made Estimators enable you to work at a much higher conceptual level than the base TensorFlow APIs.
—Martin Heller
IDGKeras
Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. Additional back ends, such as MXNet and PlaidML, are supported by third parties.
The implementation of Keras within TensorFlow, tf.keras, has some TensorFlow-specific enhancements. Keras isn’t the only simplified high-level API for building neural network models, but its rising prominence in TensorFlow emphasizes its quality and importance. Keras is currently the second-most-cited neural networks API, after TensorFlow.
Keras was created to be user friendly, modular, easy to extend, and to work with Python. The API was “designed for human beings, not machines,” and “follows best practices for reducing cognitive load.”
Neural layers, cost functions, optimizers, initialization schemes, activation functions, and regularization schemes are all standalone modules in Keras that you can combine to create new models. New modules are simple to add, as new classes and functions. Models are defined in Python code, not separate model configuration files.
The biggest reasons to use Keras stem from its guiding design principles, primarily the one about being user friendly. Beyond ease of learning and ease of model building, Keras offers the advantages of broad adoption, supports a wide range of production deployment options, has strong support for multiple GPUs and distributed training, and is backed by Google, Microsoft, Amazon, Apple, Nvidia, Uber, and others.
—Martin Heller
IDGPyTorch
A library for creating tensors and dynamic neural networks in Python with strong GPU acceleration, PyTorch is currently the third-most-cited neural networks framework, after TensorFlow and Keras. A dynamic neural network is one that can change from iteration to iteration, for example allowing a PyTorch model to add and remove hidden layers during training to improve its accuracy and generality. PyTorch recreates the graph on the fly at each iteration step.
PyTorch integrates acceleration libraries such as Intel MKL, Nvidia cuDNN, and Nvidia NCCL to maximize speed. Its core CPU and GPU tensor and neural network back ends—TH (Torch), THC (Torch Cuda), THNN (Torch Neural Network), and THCUNN (Torch Cuda Neural Network)—are written as independent libraries with a C99 API. At the same time, PyTorch is not a Python binding into a monolithic C++ framework—the intention is for PyTorch to be deeply integrated with Python and to allow the use of other Python libraries.
PyTorch has the ability to snapshot a tensor whenever it changes. The framework approximates the gradient at every saved tensor by looking at the differences between that point and the previous tensor. Called “autograd,” this speeds up calculating gradients by up to a factor of three. Given that the steepest descent optimizers rely on gradients, it can speed up the entire training process by up to a factor of three. TensorFlow has the same capability.
PyTorch is primarily supported by Facebook, but other sponsors and contributors include Twitter, Salesforce, and Microsoft. Microsoft has contributed technology that originated in its own CNTK framework, to add to the capabilities PyTorch inherited from Torch and Caffe.
—Martin Heller
IDGHorovod
Horovod is a distributed training framework for TensorFlow, Keras, and PyTorch, created at Uber. The goal of Horovod is to make distributed deep learning fast and easy to use. Horovod builds upon ideas from Baidu’s draft implementation of the TensorFlow ring allreduce algorithm.
Uber originally tried using Distributed TensorFlow with parameter servers. The engineers found the MPI model to be much more straightforward and to require far less code changes. Uber claims that the Horovod system makes it possible to train AI models roughly twice as fast as a traditional deployment of TensorFlow.
Horovod uses Open MPI (or another MPI implementation) for message passing among nodes, and Nvidia NCCL for its highly optimized version of ring allreduce. Horovod achieves 90 percent scaling efficiency for both Inception-v3 and ResNet-101, and 68 percent scaling efficiency for VGG-16 on up to 512 Nvidia Pascal GPUs.
In December 2018, Uber announced that it was moving the Horovod project under the wing of the Linux Foundation’s LF Deep Learning Foundation for open source artificial intelligence software.
—Martin Heller
IDGXGBoost
XGBoost (eXtreme Gradient Boosting) is an open source machine learning library that implements distributed gradient boosting for Python, R, Java, Julia, and other programming languages. The core project code is in C++.
XGBoost provides a parallel tree boosting algorithm that solves many data science problems in a fast and accurate way. The same code runs on single machines and distributed environments (Hadoop, MPI, Spark, etc.). The distributed version can scale to solve problems even bigger than billions of examples.
XGBoost became famous in data science circles by winning a number of Kaggle competitions. It originated in a research project at the University of Washington. The 2016 paper on XGBoost by Tianqi Chen and Carlos Guestrin explains gradient tree boosting algorithms and the refinements added to XGBoost, such as cache-aware prefetch and sparsity awareness. The paper also compares the performance of XGBoost with two other commonly used exact greedy tree boosting implementations for classification, in Scikit-learn and R.gbm, and tests XGBoost against pGBRT (parallel gradient boosted regression trees) on the learning to rank problem.
—Martin Heller
IDGH2O Driverless AI
When it comes to turning raw data into predictive analytics, H2O Driverless AI outpaces all comers with its automated simplicity. Best-practice guardrails and guideposts direct non-AI experts down a path of discovery to uncover hidden patterns using supervised and unsupervised machine learning.
You supply the data and isolate the dependent variables. H2O’s homegrown algorithms do the heavy lifting of feature engineering, model selection, training, and optimization.
It’s not magic. You’re still expected to have an understanding of your data set and the capacity to interpret the output. But H2O’s visual tools and clear explanations go a long way to bridge understanding across business teams, IT, and data scientists.
Data scientists and developers can lift the hood to finesse model parameters and build out functions with Python and R. Jupyter notebooks export machine learning pipeline code for production. Whether on-premises or in the cloud, H2O can work with your existing big data infrastructure such as Hadoop or Spark, ingest data from HDFS, Amazon S3, or Azure Data Lake, and tap Nvidia GPU processing for additional speed.
H2O pushed several significant updates in 2018—most importantly for natural language processing, time series prediction, and gradient boosting. Visual decision trees now graphically step users through understanding “how” a prediction was made—to clarify, for example, why an insurance claim was flagged fraudulent. H2O even started making its algorithms directly available on the Amazon and Microsoft clouds.
H2O Driverless AI won’t put data engineers out on the street or solve every advanced machine learning problem. But it provides a streamlined alternative to building AI from scratch, reducing the time required for businesses to become more predictive, and less reactive.
—James R. Borck