Tag: ai

Categories
Tips

Where do ML developers run their code?

In this blog post we’ll explore where ML developers run their app or project’s code, and how it differs based on how they are involved in machine learning/AI, what they’re using it for, as well as which algorithms and frameworks they’re using.

Machine learning (ML) powers an increasing number of applications and services which we use daily. For some organisations and data scientists, it is not just about generating business insights or training predictive models anymore. Indeed, the emphasis has shifted from pure model development to real-world production scenarios that are concerned with issues such as inference performance, scaling, load balancing, training time, reproducibility, and visibility. Those require computation power, which in the past has been a huge hindrance for machine learning developers.

A shift from running code on laptop & desktop computers to cloud computing solutions

The share of ML developers who write their app or project’s code locally on laptop or desktop computers, has dropped from 61% to 56% between the mid and end of 2019. Although the five percentage points drop is significant, the majority of developers continue to run their code locally. Unsurprisingly, amateurs are more likely to do so than professional ML developers (65% vs 51%).

By contrast, in the same period, we observe a slight increase in the share of developers who deploy their code on public clouds or mainframe computers. In this survey wave, we introduced multi cloud as a new possible answer to the question: “Where does your app/project’s code run?” in order to identify developers who are using multiple public clouds for a single project.

As it turns out, 19% of ML developers use multi cloud solutions (see this multi-cloud cheat sheet here) to deploy their code. It is likely that, by introducing this new option, we underestimate the real increase in public cloud usage for running code; some respondents may have selected multi cloud in place of public cloud. That said, it has become increasingly easy and inexpensive to spin up a number of instances and run ML models on rented cloud infrastructures. In fact, most of the leading cloud hosting solutions provide free Jupyter notebook environments that require no setup and run entirely in the cloud. Google Colab, for example, comes reinstalled with most of the machine learning libraries and acts as a perfect place where you can plug and play to build machine learning solutions where dependency and compute is not an issue.

While amateurs are less likely to leverage cloud computing infrastructures than professional developers, they are as likely as professionals to run their code on hardware other than CPU. As we’ll see in more depth later, over a third of machine learning enthusiasts who train deep learning models on large datasets use hardware architectures such as GPU and TPU to run their resource intensive code.

Developers working with big data & deep learning frameworks are more likely to deploy their code on hybrid and multi clouds

Developers who do ML/AI research are more likely to run code locally on their computers (60%) than other ML developers (54%); mostly because they tend to work with smaller datasets. On the other hand, developers in charge of deploying models built by members of their team or developers who build machine learning frameworks are more likely to run code on cloud hosting solutions.

Teachers of ML/AI or data science topics are also more likely than average to use cloud solutions, more specifically hybrid or multi clouds. It should be noted that a high share of developers teaching ML/AI are also involved in a different way in data science and ML/AI. For example, 41% consume 3rd party APIs and 37% train & deploy ML algorithms in their apps or projects. They are not necessarily using hybrid and multi cloud architectures as part of their teaching activity.

The type of ML frameworks or libraries which ML developers use is another indicator of running code on cloud computing architectures. Developers who are currently using big data frameworks such as Hadoop, and particularly Apache Spark, are more likely to use public and hybrid clouds. Spark developers also make heavier use of private clouds to deploy their code (40% vs 31% of other ML developers) and on-premise servers (36% vs 30%).

Deep learning developers are more likely to run their code on cloud instances or on-premise servers than developers using other machine learning frameworks/libraries such as the popular Scikit-learn python library. 

There is, however, a clear distinction between developers using Keras and TensorFlow – the popular and most accessible deep learning libraries for python – compared to those using Torch, DeepLearning4j or Caffe. The former are less likely to run their code on anything other than their laptop or desktop computers, while the latter are significantly more likely to make use of hybrid and multi clouds, on-premise servers and mainframes. These differences stem mostly from developers’ experience in machine learning development; for example, only 19% of TensorFlow users have over 3 years of experience as compared to 25% and 35% of Torch and DeepLearning4j developers respectively. Torch is definitely best suited to ML developers who care about efficiency, thanks to an easy and fast scripting language, LuaJIT, and an underlying C/CUDA implementation.

Hardware architectures are used more heavily by ML developers working with speech recognition, network security, robot locomotion and bioengineering. Those developers are also more likely to use advanced algorithms such as Generative Adversarial Networks and work on large datasets, hence the need for additional computer power. Similarly, developers who are currently using C++ machine learning libraries make heavier use of hardware architectures other than CPU (38% vs 31% of other developers) and mainframes,  presumably because they too care about performance.

Finally, there is a clear correlation between where ML developers’ code runs and which stage(s) of the machine learning/data science workflow they are involved in. ML developers involved in data ingestion are more likely to run their code on private clouds and on-premise servers, while those involved in model deployment make heavier use of public clouds to deploy their machine learning solutions. 31% of developers involved across all stages of the machine learning workflow – end to end – run code on self hosted solutions, as compared to 26% of developers who are not. They are also more likely to run their code on public and hybrid clouds. 

By contrast, developers involved in data visualisation or data exploration tend to run their code in local environments (62% and 60% respectively), even more so than ML developers involved in other stages of the data science workflow (54%).

Developer Economics 18th edition reached 17,000+ respondents from 159 countries around the world. As such, the Developer Economics series continues to be the most global independent research on mobile, desktop, industrial IoT, consumer electronics, 3rd party ecosystems, cloud, web, game, AR/VR and machine learning developers and data scientists combined ever conducted. You can read the full free report here.

If you are a Machine Learning programmer or Data Scientist, join our community and voice your opinion in our current survey to shape the next State of the Developer nation report.

Categories
Languages Platforms

The Significance of AlphaGo: Has a golden age for artificial intelligence just dawned?

artificial intelligence

In recent years artificial intelligence (AI) has returned to the forefront of technological debate. That debate has moved on from when, and even whether, computers will ever display intelligent behaviour to how smart they will get, how quickly, and what the implications are for society. Although there are multiple approaches to creating AIs, the ones that involve machine learning from large datasets are generally outperforming all others. The results from such systems are often so impressive that large companies are rushing to hire data scientists, collect more data, and apply the latest machine learning techniques to inform their management decision making. Google’s DeepMind team recently demonstrated that without any human in the loop they can build a system that makes complex strategic decisions better than a human expert. Their approach suggests a way forward for building such systems in many diverse fields.

Machine_learning&artificial_intelligence

The game computers couldn’t beat

The announcement from the DeepMind team that their AlphaGo program had defeated the European champion at the game of Go was a highly significant landmark in AI. Not only did they accomplish a long-standing ‘grand challenge’ in AI and surpass rival Facebook’s efforts by an enormous distance, but the way the system works is in many ways very human-like. At first glance it’s easy to dismiss game-playing AI systems as not immediately applicable to real-world problems. The ‘world’ the AI operates in is incredibly simple compared to our physical world – in the case of Go, a 19×19 board where a black or white stone can be placed on each intersection. However, [tweetable]advances in AI from the pursuit of better Go playing programs are already being used[/tweetable] in real-world applications elsewhere. Also, the ‘deep convolutional neural networks’ that AlphaGo uses to ‘perceive’ the board are similar to those currently being employed to push forward the state-of-the-art in image and speech recognition, as well as natural language processing.

It’s different this time

Back in 1997, IBM’s Deep Blue beat the world Chess champion, Garry Kasparov. How is this different? First, Go is significantly more complex than Chess. There are nearly an order of magnitude more moves possible from every position and each move can have a bigger impact on the strength of a player’s position. Second, Deep Blue used a supercomputer and some hand-crafted heuristics to effectively do a brute force search of all reasonable future move combinations to pick the best move to make next. This was nothing like the way a human would play Chess and also not generalisable to other problems.

In contrast to Deep Blue, AlphaGo combines two deep convolutional neural networks with a Monte Carlo Tree Search algorithm to select moves in a way that’s quite similar to the way a human would play. The first neural network, called the policy network, picks a few promising positions for the next move. A human player doesn’t systematically evaluate all possible moves, rather through experience they develop an intuition for moves that should make their position stronger. They would struggle to explain why they selected a specific move over others in many cases. This suggests they’ve developed a model for how to play that exists below their conscious awareness. AlphaGo’s policy network is trained to predict the moves that expert players would make using a dataset of 30 million different positions from real games. The second neural network, called the value network, estimates how strong any given position is. It was trained, simplifying slightly, using the results of the policy network alone playing against itself from 30 million distinct positions. The Monte Carlo Tree Search is then used to look ahead from each move selected by the policy network at the opponent’s likely responses and AlphaGo’s subsequent moves. However, rather than search all the way to the end of the game, the value network is used to evaluate the end position after a sequence of moves. This is also similar to human play, looking a handful of moves ahead to assess the probability of gaining an advantage with each possible move. The lookahead searches are shallow (constrained by the processing power and time allowed for a move) and yet the results are better than existing leading systems that look much further ahead but with much less sophisticated move candidate selection and position evaluation capabilities.

Widely applicable artificial intelligence

This might all still sound a long way from a truly human style of thinking but if we abstract and generalise it slightly then it becomes more familiar. For any goal-oriented behaviour in a complex or changing environment we can assess our current situation versus our goal and generate some options for moving towards the goal. We can then simulate or predict the results of taking those actions and evaluate the new situations we could get into. We choose the option that moves us closest to our goal, or has the highest probability of moving us closer to that goal. This is just a description of iterative planning.

AlphaGo has shown that we can train a machine-learning system to emulate the options a human would select in a relatively-complex environment. If we simulate the immediate results of those selections we then just need to evaluate where we get to with each option. Again: machine learning comes to the rescue. If we can acquire or generate enough data we can train another machine learning system to perform the evaluation. None of this is really a new idea but now it has been demonstrated to be good enough to beat a professional at Go, it’s a fair bet it can be made to work for a huge range of other problems too. This is possibly why it’s such a landmark for AI research. It’s a challenge that until very recently was thought to require a completely new breakthrough in AI and probably another decade of research (and Moore’s Law) to get us there. It turns out the techniques we’ve already invented, when suitably combined, can achieve very intelligent behaviour.

Dawn of a golden age?

There’s an outside chance Go just happened to be a lot easier than we thought, or just unusually suited to these ‘deep learning’ techniques. However, given the progress that’s being made with deep learning on other longstanding AI problems it seems more probable that [tweetable]we’re about to enter a golden age for AI[/tweetable]. In this context it’s interesting to note that AlphaGo beat the European champion a month before Google opened the source code for their TensorFlow deep learning framework (which prompted Microsoft to follow suit with theirs). These open source moves can be seen as part of a land grab for talent and mindshare. The techniques are the subject of published research and efficient implementations are valuable but nowhere near as much as the data required for training and the talent to utilise it. Then of course, Google, Microsoft, Amazon and a bunch of startups will all offer managed solutions for training and running these kinds of framework on their clouds. As the significance of DeepMind’s accomplishment sinks in, more researchers and developers will rush to jump on the machine learning bandwagon and there will be no shortage of tech giants waiting to welcome them aboard.