Machine Learning Basics: Components, Application, Resources and More – Part I

Articles From: QuantInsti
Website: QuantInsti

Machine learning has become a hot topic today, with entrepreneurs all across the world switching to machine learning for business operations. Machine learning has reached the advancement where it can even predict outcomes without being explicitly programmed to do so.

This is not only it, but there is a lot more when it comes to the applications of machine learning in trading. With this blog, you will learn all about the basics of machine learning and how to begin learning the same along with the resources for learning, the applications of the same and much more!

Dive into this interesting blog which covers:

  • What is machine learning?
  • Example of machine learning
  • History of machine learning
  • Importance of machine learning
  • Components of machine learning
  • Machine learning classification
  • Difference between machine learning and deep learning
  • Prerequisites to learn machine learning
  • Python libraries for machine learning
  • Common terms used in machine learning
  • Application of machine learning in trading
  • Resources to learn machine learning
  • The future of machine learning

What is machine learning?

Machine Learning, as the name suggests, provides machines with the ability to learn autonomously based on experiences, observations and analysing patterns within a given data set without explicitly programming.

When we write a program or a code for some specific purpose, we are actually writing a definite set of instructions which the machine will follow.

Whereas in machine learning, we input a data set through which the machine learns by identifying and analysing the patterns in the data set. Then, the machine will make decisions autonomously based on its observations and learnings from the dataset.


Example of machine learning

Although there are numerous examples of machine learning, we are covering just a few here.

  • Facebook: For instance, think of Facebook’s facial recognition ⁽¹⁾ algorithm which prompts you to tag photos whenever you upload a photo.
  • Alexa, Cortana, and other voice assistants: Another example is of the voice assistants who use machine learning to identify and service the user’s request.
  • Tesla automobiles: One more example is of Tesla’s autopilot ⁽²⁾ feature.

Now let us see an example of a “bird species recognition learning” problem. This example is explained with the task of the model, the performance measure of the model and the training experience required for the accurate results:

  • The task of the machine learning model: Recognizing and classifying species of birds within images
  • Performance measure: Percent of bird species correctly classified
  • Training experience of the machine learning model: Training on a data-set of bird species with given classifications

Hence, the machine learning model will learn the task according to the performance measure and the required training experience of the machine learning model.


History of machine learning

Machine learning is not a recent phenomenon. In fact, neural networks were first introduced in the year 1943 ⁽³⁾!

Although in the early days, progress in machine learning was somewhat slow due to the high cost of computing. The high computing cost made this domain only accessible to large academic institutions or multinational corporations. Also, the data in itself was difficult to acquire for a company’s needs.

But with the advent of the internet, we are now generating quintillions of data everyday ⁽⁴⁾!

Couple that with the reduction in the price of computations and we find that machine learning is more than a viable proposition ⁽⁵⁾.

Some of the notable events in the history of machine learning are:

  • 1950 – This was the first time when “Alan Turing” ⁽⁶⁾ created a test in order to check if a machine could fool a human being into believing that it was talking to a machine.
  • 1952 – The first computer learning program, a game of checkers, was written by Arthur Samuel.
  • 1957 – The first neural network for computers was invented by Frank Rosenblatt, which simulated the thought process of a human brain.
  • 1967 – The Nearest Neighbor algorithm was written.
  • 1979 – Students of Stanford University, California invented the Stanford cart which could navigate and avoid obstacles on its own.
  • 1997 – IBM’s Deep Blue beats the world champion at Chess.
  • 2002 – A software library for machine learning named Torch was first released.
  • 2016 – AlphaGo algorithm developed by Google DeepMind managed to win five games out of five in the Chinese Board Game Go competition.

These events can be represented as:

Importance of machine learning

Machine learning plays an important ⁽⁷⁾ role in the field of enterprises as it enables entrepreneurs to minimise manual efforts. The machine learning model learns with the help of humans but eventually, the machine learns and takes over the learnt task.

Although a minimum level of intervention is needed for making sure that no “machine-related” glitch arises or for updating the data inputted.

Nowadays leading companies like Google, Amazon, Facebook, Tesla, and many more are efficiently utilising these technologies. Hence, machine learning is proving to become a core part of operation and functioning.

Moreover, there are a multitude of use cases that machine learning can be applied to in order to cut costs, mitigate risks, and improve overall quality of life including risk management. Furthermore, the global machine learning (ML) market is expected to grow ⁽⁸⁾ from $21.17 billion in 2022 to $209.91 billion by 2029, at a CAGR of 38.8% in forecast period.


Components of machine learning

There are tens of thousands of machine learning algorithms and hundreds of new algorithms are developed every year.

Every machine learning algorithm has three components:

  • Representation: This implies how to represent knowledge. Examples include decision trees, sets of rules, instances, graphical models, neural networks, support vector machines, model ensembles and others.
  • Evaluation: This is the way to evaluate candidate programs (hypotheses). Examples include accuracy, prediction and recall, squared error, likelihood, posterior probability, cost, margin, entropy k-L divergence and others.
  • Optimization: Last but not the least, optimization is the way candidate programs are generated and is known as the search process. For example, combinatorial optimization, convex optimization, and constrained optimization.

All machine learning algorithms are a combination of these three components and a framework for understanding all algorithms.

Machine learning classification

Machine Learning algorithms can be classified into:

  1. Supervised Algorithms:
    ◦ Linear Regression,
    ◦ Logistic Regression,
    ◦ KNN classification,
    ◦ Support Vector Machine (SVM),
    ◦ Decision Trees,
    ◦ Random Forest,
    ◦ Naive Bayes’ theorem
  2. Unsupervised Algorithms: K Means Clustering
  3. Reinforcement Algorithm

Let us dig a bit deeper into these machine learning basics algorithms.

Supervised Machine Learning Algorithms

In this type of algorithm, the data set on which the machine is trained consists of labelled data or simply said, consists of both the input parameters as well as the required output.

Let’s take the previous example of facial recognition and once we have identified the people in the photos, we will try to classify them as babies, teenagers or adults.

Here, babies, teenagers and adults will be our labels and our training dataset will already be classified into the given labels based on certain parameters through which the machine will learn these features and patterns and classify some new input data based on the learning from this training data.

Supervised Machine Learning Algorithms can be broadly divided into two types of algorithms; Classification and Regression.

Classification Algorithms

Just as the name suggests, these algorithms are used to classify data into predefined classes or labels. We will discuss one of the most used classification algorithms known as the K-Nearest Neighbour (KNN) Classification Algorithm.

Regression Machine Learning Algorithms

These algorithms are used to determine the mathematical relationship between two or more variables and the level of dependency between variables. These can be used for predicting an output based on the interdependency of two or more variables.

For example, an increase in the price of a product will decrease its consumption, which means, in this case, the amount of consumption will depend on the price of the product.

Here, the amount of consumption will be called the dependent variable and the price of the product will be called the independent variable. The level of dependency on the amount of consumption on the price of a product will help us predict the future value of the amount of consumption based on the change in prices of the product.

Unsupervised Machine Learning Algorithms

Unlike supervised learning algorithms, where we deal with labelled data for training, the training data will be unlabelled for Unsupervised Machine Learning Algorithms. The clustering of data into a specific group will be done on the basis of the similarities between the variables.

Some of the unsupervised machine learning algorithms are K-means clustering and neural networks.

A simple example would be that given the data of football players, we will use K-means clustering and label them according to their similarity. Thus, these clusters could be based on the striker’s preference to score on free kicks or successful tackles, even when the algorithm is not given pre-defined labels to start with.

K-means clustering would be beneficial to traders who feel that there might be similarities between different assets which cannot be seen on the surface.

While we did mention neural networks in unsupervised machine learning algorithms, it can be debated that they can be used for both supervised as well as unsupervised learning algorithms. Artificial neural network and Recurrent Neural networks also fall under unsupervised machine learning algorithms.

Reinforcement Machine Learning Algorithms

Reinforcement learning is a type of machine learning in which the machine is required to determine the ideal behaviour within a specific context, in order to maximise its rewards.

It works on the rewards and punishment principle which means that for any decision which a machine takes, it will be either rewarded or punished. Thus, it will understand whether or not the decision was correct.

This is how the machine will learn to take the correct decisions to maximise the reward in the long run.

For a reinforcement algorithm, a machine can be adjusted and programmed to focus more on either the long-term rewards or the short-term rewards. When the machine is in a particular state and has to be the action for the next state in order to achieve the reward, this process is called the Markov Decision Process.

Stay tuned for the next installment in this series to learn about the difference between machine learning and deep learning.

Visit QuantInsti for additional information on this topic: https://blog.quantinsti.com/machine-learning-basics/.

Disclosure: Interactive Brokers

Information posted on IBKR Campus that is provided by third-parties does NOT constitute a recommendation that you should contract for the services of that third party. Third-party participants who contribute to IBKR Campus are independent of Interactive Brokers and Interactive Brokers does not make any representations or warranties concerning the services offered, their past or future performance, or the accuracy of the information provided by the third party. Past performance is no guarantee of future results.

This material is from QuantInsti and is being posted with its permission. The views expressed in this material are solely those of the author and/or QuantInsti and Interactive Brokers is not endorsing or recommending any investment or trading discussed in the material. This material is not and should not be construed as an offer to buy or sell any security. It should not be construed as research or investment advice or a recommendation to buy, sell or hold any security or commodity. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.