This tweet was doing the rounds the other day:
Still the simple best explanation of what machine learning is compared to classical programming. pic.twitter.com/grHOIxoW3y— Thibaut (@Kpaxs) August 18, 2019
I have conflicting feelings about this figure - I find it to be at the same time incomplete, misleading, and insightful.
It being incomplete is to be expected for any simplistic representation. Most obviously, this visual is focused only on the training phase of supervised learning. Machine learning algorithms can generate rules even without answers (unsupervised learning) and eventually, the rules do also need to be fed back into code that, together with new data, generates answers (prediction phase).
It is misleading because the proposed dichotomy between the ‘classical programming’ and the ‘machine learning’ paradigm is a false one. Some points.
So-called ‘classical programming’ is itself a container of different paradigms. Not all of them are aimed at producing answers from data through rules. Common alternative goals are simply codifying a desired result, or mutating the current state of data into a new state, with or without additional inputs.
Machine learning code will usually be embedded in a script with explicit instructions (rules) on which steps to perform, and with which parameters to perform them.
On a deeper level, machine learning algorithms are themselves a set of rules on how to generate new rules from data.
The decision whether to use ‘classical programming’ or ‘machine learning’ is not one of choosing a more modern approach, but is dependent on the problem at hand and on the availability of relevant data and/or answers. Only a small subset of problems that previously might have been solved with manually defined rules, can now be solved with machine learning.
This visual reinforces the increasingly prevalent idea that machine learning is a new subdiscipline of computer science, when it is really a relatively old subdiscipline of statistics, which happens to be powered by computationally intensive algorithms and very forgiving on the statistical innocence of people educated in software engineering.
Expanding on the previous point, one might posit that all of science concerns the derivation of rules from data, and all of engineering concerns making these rules actionable for practical purposes. Machine learning then adds to this centuries old process the rule-based automation allowed by ‘classical programming’. So which is a paradigm shift for which?
Now that this poor tweet lies trampled in the dust, let me say that I do find it insightful - and that I would use and share it professionally. One of the most important lessons to learn as a data scientist is that business clients don’t care for what we offer most abundantly - caveats, nuance and context. They want bite-sized, actionable insights so they can do their jobs: Make decisions, and make things happen.
In the eyes of an outsider, machine learning is ‘something new that the techies do’. So, how then is it different from their previous work? Well, they tame the data beast that we have been increasingly struggling with over the years, and let it write useful code in their stead. As a dichotomy this is wildly inaccurate, but it does drive home the main message.
And all demystification is good. Ergo, retweeted!