Machine learning is revolutionising business in the 21st century. Whilst people are aware of machine learning in its most obvious forms, most are unaware of the sheer variety of techniques that can be applied to improve businesses. Whilst neural networks dominate the news and popular culture, they are only one of a range of techniques applied by machine learning engineers.
A common way in data science to categorise AI techniques is according to the training data used. Training data is the data set which data scientists use to train their models. Supervised learning uses labelled training data. For example, if we were comparing the characteristics of stroke patients to healthy individuals, each individual’s health information would have a corresponding label of having had a stroke or not. Unsupervised learning, on the other hand, involves unlabelled data, relying on techniques to make sense of the inherent structure of a dataset without external guidance.
Unsupervised learning is used to derive insight into the innate structure of data-sets to drive data-analytics. There are two main areas of analysis.
The first, clustering, separates out members of a dataset into groups using statistical analysis to find features common to each group. It is most commonly used to supercharge consumer segmentation by looking at consumer behaviour and demographics. This guides precision strategy by allowing companies to target each of these groups individually.
There are different types of clustering. Some clustering techniques straightforwardly place individuals into categories, whilst probabilistic methods give probabilities associated with an individual’s membership to each group. The second is valuable, for example, in scenarios where marketing to individuals is sufficiently low cost that a mixture of ads pertaining to each group can be given to borderline members. Additionally, hierarchical clustering can be used to make several levels of subdivisions within groups. This has the additional advantage of allowing companies to flexibly target groups at the appropriate level of specificity.
The second type of unsupervised learning is association learning, which looks for rules governing relationships between items in a dataset. This is the kind of technique used to generate recommended products or contents on online platforms. Algorithms recommend products having learnt the relationship between the first product and the second, that people who interact with the first will engage with the second. It can use simple association rules comparing the probability of interacting with the second given the first, or more complex algorithms to determine these relationships.
These learning techniques dramatically improve engagement and sales on online platforms by recommending relevant content or can alternatively be used to extract insight into consumer preferences to guide strategy.
Unsupervised learning has certain drawbacks, however. It struggles with large datasets and is tricky to guide without external input – there is no guarantee that the same segmentation of the data happens on different occasions.
Supervised learning is also divided into two main categories. Categorisation involves sorting new data into discrete categories based on a labelled training set. This has many applications in fields that need to make high-quality decisions. For instance, according to a 2019 review of studies, machine learning has been shown to be comparable to experts in categorising clinically significant prostate cancer via MRI analysis. In industry, categorisation can automate numerous tasks such as labelling documents, product quality control, or determining whether customers are likely to purchase a given product.
There are many techniques used for categorisation. Random forests take the majority view of several decision trees trained on labelled data. Decision trees are effectively rule-based tests learnt automatically off a data set which predict which group you belong to. There are many rules which an algorithm could arrive at depending on how you set them up, and they can make different decisions about novel items. By taking the majority view of many different decision trees can create a more accurate categorisation procedure.
Alternatively, another popular option is logistic regression. For example, if we are predicting whether an individual will purchase a product based on demographic and behavioural information, logistic regression analyses the relationship between each variable and the likelihood of purchasing the product. These relationships are then used to predict the likelihood in new instances.
More commonly, regression is used to predict a continuous variable. The term regression typically refers to predicting an outcome associated with a data item. For example, regression can be used to predict the expected lifetime value of a customer by comparing their characteristics with past consumer data. It can also be used for planning and forecasting – for instance, a franchise could reduce food waste by using previous years’ sales data to predict sales performance for a given week, thereby more closely matching stock orders to actual usage. This type of supervised learning can optimise the distribution of financial and human resources.
It is also necessary to mention the role of neural networks in supervised learning. Neural networks are capable of both supervised and unsupervised learning, though the most popular models today perform supervised learning. Neural networks are loosely modelled on the human brain and consist of neurons arranged in layers. Neurons are connected between layers by weighted connections, and the activity in later neurons is determined by the activity of previous layers and the weighted connections between them. These weights are adjusted to reduce the error in the network’s output, such as predicting a value in labelled data.
By learning a function that predicts the outcome for each data item, the network can have novel data items input and hopefully correctly predict their outcome correctly. Whilst often more computationally intensive, and requiring large amounts of data, neural networks are often capable of outperforming alternative algorithms in their accuracy. This has opened up more and more regions in which supervised learning can be used to transform company processes, augmenting, and automating existing processes whilst also opening up novel avenues for growth.
Given the wide range of strategic opportunities, it is crucial to understand the various analytic techniques available. The often-cited “no free lunch” theorem argues that no algorithm is universally superior when applied to all possible problems. The takeaway is that there is no one-size-fits-all solution. Instead, it is essential to remain flexible and aware of the many different techniques and their applications.