Financial Engineering and Artificial Intelligence in Python - TLP Udemy

1. Welcome

1. Introduction and Outline

Hey, everyone, and welcome to Financial Engineering with artificial intelligence and Python. So this course might come as a big surprise to a lot of you. In my 30 or so previous courses, I focused solely on the topics of machine learning and data science. I have in until this point looked at any specific industry which applies machine learning and data science, whether that be biology, finance or social networks. And if you think about it, that kind of makes a lot of sense. If you start talking about finance, the biology student won’t understand. If you talk about biology, the finance student won’t understand. So I think it’s more fair to students to talk about machine learning in a more general sense. You learn the machine learning algorithms and you apply them to whatever industry you come from. In this course, we are going to drill down into a specific domain and that domain is a financial engineering marvel. So I think students of machine learning and they are the best candidates for learning financial engineering. As you will come to learn in this course, many of the techniques we will use, you will already be familiar with from your studies in machine learning. So that’s going to be the underlying perspective in this course. What does financial engineering look like to someone who knows at least the basics in machine learning? By taking this approach, this will let us go more in depth and gain a better understanding compared to, say, someone just approaching finance for the first time. Of course, if you are not a student of machine learning, that’s OK, too. If you clicked on this video, it’s probably because you were looking for the words financial engineering and if you were, then you are in the right place. This, of course, is like financial engineering on steroids. Instead of just being plain old financial engineering, it’s financial engineering with lots of extra bonus machine learning sprinkles. OK, so what are we going to learn about in this course? This, of course, is going to start from some very basic fundamentals. What is financial data? Where do we get it and what does it look like? How do we manipulate it in code? In particular, this course we’ll focus on a stock prices and stock returns. We will spend a lot of time learning how to analyze the distribution of stock returns and how they might be correlated with other stock returns. In the next section, we will go in-depth into Time series analysis. Now, in all my years of machine learning, I’ve noticed that one of the most common questions students ask is how can I predict stock prices with machine learning? This section will give you a very detailed answer to this question, more detailed than you even thought you could care to know. This section will look at various core techniques in Time series analysis, such as the simple and exponential moving average, the whole winters’ model and the famous Arima model. Since this is, of course, about finance. There will be some finance specific details you want to be aware of. Specifically, we’ll learn about the efficient market hypothesis and the random walk hypothesis. We’ll learn about what implications these have on the models we create. One of the most important topics in this section is how to properly forecast. Unfortunately, the Internet today is full of marketers. These marketers have promised to teach you how to do things like predict stock prices with LSD and they show you charts like this with nearly perfect stock price predictions. Now, you’ve got to wonder if I can just do this. Why did we give a guy a Nobel Prize for inventing portfolio theory? If we can just predict stock prices, we don’t need portfolios. Let’s just pick the stock with the highest forecast and make lots of money. In this course, I will expose the many mistakes in these courses in blogs. If you’ve ever come across a course or a blog post about predicting stock prices with storms, you will want to watch this. OK, so the next section of the course is going to be about modern portfolio theory. This section is the financial engineer’s version of Don’t put all your eggs in one basket. In this section, we’ll learn about why diversification is good and how it weakens our greatest enemy, which is risk. You’ll learn about core topics in financial engineering, such as the efficient frontier, mean variance optimization, the Sharpe Ratio and the capital asset pricing model. Basically given a universe of stocks that is any set of stocks you want to consider buying. We will learn how to maximize return while at the same time minimizing risk. We will learn how investing in multiple stocks will always be better than investing in just a single stock. We can get higher returns and lower risk. OK, so in the next section, we are going to learn about algorithmic trading previously when we looked at portfolio theory, it was all about which stocks to buy and which stocks to sell. This is all about determining when is the best time to buy and when is the best time to sell. As you probably already know, if you tried to use your emotions, you’re going to have a bad time. When people use their emotions, they tend to buy high and sell low. They see a stock going up and they think it’s a great deal. They see a stock going down and they get scared and try to cut their losses. Unfortunately, this is the exact opposite of what we logically know we should do. We want to buy low, sell high. This results in a profit. Algorithmic trading is all about how to automate buying and selling decisions so that you do not depend on your emotions, which are probably wrong. We will look at both traditional trading strategies as well as those based on machine learning and reinforcement learning. All right, so I hope that gives you a pretty good idea of what we will look at in this course, just to summarize, first, we will start by understanding financial data, specifically stock prices and stock returns. Next, we will look at TIME series analysis, how to stock prices and stock returns evolve over time. Next, we will look at portfolio theory. How do we choose the right set of stocks in the right proportions such that we maximize, return and minimize risk? Next, we will look at algorithmic trading. How do we automate the process of buying and selling to avoid trading on emotion? Thanks for listening and I’ll see you in the next lecture.

2. Where to get the code

In this lecture, I’m going to describe where to get the code for this cause, the code for this course will be available from two places. These two places are as follows. Number one, Google CoLab. These are like Jupiter notebook’s, but 1000 times better. They are hosted in the cloud and they give you free access to GPS to accelerate model training. Number two, GitHub get is a technology that every developer today should know about, whether you are a machine learning engineer or an iOS developer. So let’s first talk about Google CoLab, if you see a code lecture that looks like this, you can see that it looks like a Jupiter notebook. This will indicate that we are using Google CoLab. You can identify which notebook we are looking at by its title. There is a resource that you can find in the extras section of this course which will provide you with the link for all the notebooks in this course. Again, if you see a column notebook, then you can find a link to that corresponding notebook by going to the extra section of this course. OK, number two, GitHub, when you see code like this, you can see that it’s not a Jupiter notebook. This is just a plain text editor. When you see this, that means this code is not on Google CoLab, but rather is provided as a plain text python file. This code can be found on my GitHub repository at GitHub dot com slash lazy programmer slash financial engineering. The GitHub repository also contains extra reading materials for this course, which you can find in the file. Extra reading tea. Note that for some students who really dislike having to type anything, you do not have to type this whole euro all by yourself. One alternative is to simply search for the repository on your favorite search engine, such as Google, Bing or Duck Duck go. However, please note that if typing a few characters is too much work for you, then you will probably be very surprised at how much work is involved with this course. Actually, getting the code is zero effort in relation to the course itself. So just keep that in mind and hopefully you are ready to work hard. So just to make that super, super clear, when you see a Google CoLab notebook, you’ll want to go to the extra section of this course where you will find a link to all the CoLab notebooks in this cause. When you see a plain text Python file, you’ll want to go to my GitHub repository for a copy of that same code. However, I do not recommend that just taking my code and hitting the play button and not ever coding by yourself. This is strongly discouraged. One thing I want to emphasize is that it does not matter if you prefer Google CoLab or if you prefer plain text Python files, as long as you have at least beginner level experience with Python, you should be able to take a plain text python file and put that into a CoLab notebook with zero effort. You should also be able to do the reverse. If you like plain text python files, then you should be able to take the code in a CoLab notebook and paste that into a python file. Again, this should be zero effort in relation to the actual work that you will be doing in this course. In order to get the code, I would strongly recommend using the git clone command, I repeat, use the git clone command. If you’re having difficulty using the git clone command, then as a last resort, you can simply use the download button. One thing I really want to stress, if you really, really don’t know how to use GitHub and you really, really don’t like the idea of being forced to learn to use GitHub, you are not required to just use the download button. Using the download button is like downloading any other file from the Internet. So if you can use the Internet, then you can do this. Here’s what you should not do that I’ve seen students do in the past, first, do not go to file and say that this is not how you download code from GitHub. Secondly, please do not forget the repository, I update this repository very often and you will miss these updates if you forget the repo. Furthermore, many past students have been confused by folks Reppas in the past thinking they had found my repo when in fact they had found someone else’s out of date fork. For the sake of your fellow students, please do not forget the repository. And if you have a fork, please either update it or delete it. Thirdly, if you don’t know what yet or GitHub is and you have no clue what I’m talking about, please educate yourself on why version control is absolutely 100 percent necessary for anyone who writes code. This applies whether you are in machine learning or you are an iOS developer. If you don’t know why version control is not just necessary but a superior way of storing and distributing code, please do a Google search and read about why this is true. Since I’m not here to teach you the basics of how to use a computer, you will have to take responsibility to make sure that you can install and use gear. And guys remember, using it is not some crazy new thing. Anyone with a job doing software development already knows how to use get. It would be in your best interest to ensure that you have this basic skill. I want to remind you that being hands on and engaging with the material is your responsibility. When it’s time to code, you must code, you must put your hands on the keyboard and implement yourself the algorithms we learned about. I won’t remind you that you should code, since I expect everyone taking this course to be mature enough and disciplined enough to write code when it’s time to do so. Simply copying what I typed is not encouraged. Simply downloading my code and hitting the play button is not encouraged. If you want more information, watch the lecture. How to code by yourself, which can be found in the appendix. You’re also invited to watch the lecture. Should you code along on my YouTube channel? The answer to this question is no. My motto has been and always will be. If you can’t implement it, then you don’t understand it. I’ve even had some students implement what they learned in Java. Remember that the code I provide is merely a guideline. It doesn’t matter if I use a different version of Python, if I use a different library or even a different language, be like a Java friend and code yourself. Please keep in mind that this course is always going to follow the same basic pattern. First we discuss the theory and then we discuss the code. Obviously, the theory involves the reasoning behind why and how an algorithm works. The code lecture is simply a translation of that theory into code. If you just remember this basic fact, you will never get lost or forget where you are in the course, because obviously this is pretty straightforward. First, we discuss what we are about to code and then we actually code it on this topic. Please do not randomly skip around in the lectures. Students sometimes try this and it’s obvious when they have because they ask questions that were answered in previous lectures. The lectures are in the order they are in for obvious reasons. So unless you already know what you’re doing, you probably don’t want to jump around or treat the lectures like a reference. Continuing on this theme, let’s remember that since you are the one doing the coding, you can choose whatever environment you like in which to do that coding. Sometimes people ask me, why don’t you use Jupiter notebook? Obviously, it doesn’t matter what I use because it’s in a video. The only difference between one environment and another, if it’s in a video, is that it might be a different color, which is obviously irrelevant to you. I’m not saying you should do this, but if you did not want to write any code and you just wanted to run my code, you could simply copy and paste it into a Jupiter notebook. Remember, there’s no such thing as different versions of Python for different environments. Python code works the same in all environments. It doesn’t matter if you use notebook, pi, charm, spider, sublime, text, atom or vim. It simply does not matter. You can code anywhere you like. The most important thing is that you do it in the first place.

3. Scope of the course

In this lecture, I want to talk about the scope of this cause, basically, if you want to summarize this lecture into one sentence, it’s this financial engineering is not just a single course. Financial engineering is a degree. So what do I mean by that? Well, what is a degree? A degree is usually a multiple year program that you would take at a college or university. A degree consists of multiple courses. For example, if you’re going to become a computer scientist, then you’re not going to just take a single course called computer science. You’re going to take multiple courses on different areas of math, different areas of programming, some algorithms, maybe some computer graphics compilers, operating systems and so on. Similarly, financial engineering is a very broad topic that you can spend multiple years studying. So why do I mention this? Well, I want you to realize that the scope of this course will be limited to only what you see in the course description, I can virtually guarantee that there is some topic in finance that you care about that we are not going to cover. So I want you to take this opportunity and double check the curriculum and make sure that these are topics you want to learn. Finance is a huge topic. And if you really, really want to learn something that’s not discussed in this course, I would urge you to seek it out on your own. I fully support and encourage you in doing so. It’s better for you and it’s better for me. OK, so the next thing I want to talk about in this lecture is the phrase artificial intelligence. I want you to realize that everybody is going to have their own slightly different definition of this phrase. Because of this, your personal definition of artificial intelligence is not relevant in this cause. Some people might think A.I. means deep learning. Some people might think A.I. means reinforcement learning. Some people might think A.I. means video game character. Some hardcore statisticians might consider nothing to buy and that everything we do in machine learning is just statistics repackaged for computer scientists. In this course, the phrase artificial intelligence describes the mindset one should have when taking this course. In every section of this course, we will be applying various techniques from the field of A.I., including supervised learning, unsupervised learning, convex optimization and optimal control. In other words, this course is very eccentric. At the same time, if you have a very rigid, unwavering view of AI and you really, really think your definition of AI is the only correct definition, then again, I would strongly recommend that you check the curriculum and make sure that the topics that you want to learn are in fact covered within the scope of this course. This cause really is the combination of two very exciting topics, financial engineering and artificial intelligence. But at the end of the day, it’s still just one cause there’s bound to be some finance people who want more finance and there’s bound to be some people who want more. I mean, if you recognize yourself to be either of these people, then I wish you the best of luck on your journey. Please note that the entirety of this course is for educational purposes only. It is not intended to be investment advice. If you require investment advice, please seek a licensed professional in your specific jurisdiction.

4. How to Practice

So in this lecture, we’re going to talk about how to practice everything you learned in this course. Guys, I want you to realize that this course is one of the easiest out of all my courses to practice what you’ve learned in each section of the course. I’m going to teach you a process. For example, in the first section, I’m going to show you how to analyze the distribution of a stock return. In the second section, I’m going to show you how to model stock prices and stock returns as a time series. You’ll learn how to test whether a stocks price depends on its previous values. Then if it does, you’ll be able to use that information to forecast into the future. In the third section, I’m going to show you how to create a portfolio out of a set of stocks, you’ll learn what it means for a portfolio to be optimal and how to create such an optimal portfolio yourself. Finally, you’ll learn how to build and test a trading algorithm. This is where you let a computer program decide when to buy and sell a stock. OK, so in all these cases, obviously, we’re going to have to work with some data set. However, it’s very likely that there are data sets you are interested in that we haven’t looked at. Remember that in each of these sections, what I’ve shown you is a process of showing you how to do a certain kind of analysis, how do you practice that? How do you make sure that you know what you’re doing? Well, you should make sure that you can replicate that process. How do you do that when you pick a security that you care about? Maybe that’s Bitcoin, maybe you don’t live in the US. So you care more about stocks and other markets, such as the European market or maybe the Asian market. Luckily, all you have to do is follow my rule. All data is the same. What does that mean? It means that all the code I’ve shown you in this course will work no matter what data you plug in. That means you can take a set of stocks from the European market and find an optimal portfolio using the exact same process I’ve shown you. To contrast, this is way easier than it is in my machine learning courses in these courses. The basic idea is the same. I show you some algorithm like Kenia neighbors or a boost, and you apply that to your own data sets. You get to choose which data set you care about and replicate the process that you learned. Now, what’s funny is this often makes students kind of uncomfortable. They prefer to be told which data sets to work on. But I can’t tell you what to care about. If you care about biology, then you should use a biology data set. If you care about online advertising, then you should use an online advertising data set. But don’t wait for me to tell you what to care about. Everybody cares about something different. So that’s something you have to decide for yourself. In this course. Your choices are much more limited because this course focuses on a specific domain, which is finance. So it should be clear that you’re going to choose some kind of financial object where the data looks like what we are analyzing in this course. So just to make that super, super clear, how do you practice what we learned in this course? Make sure you can do everything we do in this course on your own financial data set. You get to decide what kind of data you care about. What I’ve shown you in this course is the process. And this same process applies to the kinds of financial data that you should choose.

5. Warmup (Optional)

OK, so in this lecture, we are going to do a little warm up. So how can you think of this lecture? This is just like warming up when you do exercise. The goal is to get you primed for specific movements and for the specific ways of thinking that we will use in this course. It could also give you some sense of the kinds of tools that we will use in this course and the level of complexity you should expect. Note that if you’ve taken courses with me before, you can comfortably skip this lecture. OK, so let’s begin. So your first exercise is to use an MP to generate 1000 random samples from the standard normal, that is samples from the normal distribution with mean zero and variance. One note that this lecture will also help us to introduce some notation. You’ll see that I’m using this unconventional letter n to mean the normal. This is so that we don’t get it confused with the regular N, which I usually use to mean the number of samples in a data set. OK, so once you’ve generated these samples, you want to plot the samples as a time series. The next step is to plot the samples as a histogram. All right, so for the next exercise, you’re going to add a trend line to the noise that you just generated. Once you’ve done so, it should look something like this. Note that this is a scatterplot, not a line chart. I’ll let you pick the parameters of the trend line. So play around with different values until you get something that looks reasonable as a bonus exercise. You can try finding and plotting the line of best fit to this data, although we’ll be going through that in this course anyway. The next exercise is to call the comm some function on your noise, which calculates the cumulative some of the array that you pass in. Next, I want you to take that output and plot it as a time series. Consider what kind of time series this reminds you of. In the context of this course, the answer should be pretty obvious. But think about how what you just did might be related to what we will learn in the coming sections. In the next exercise, I want you to generate 1000 samples from the multivariate normal, this will have the main vector zero zero and the covariance matrix, one minus zero point five, minus zero point five and two. You can also use Sipi for this or any other library you prefer. Once you have your samples, I want you to plot them using a scatterplot. Again, this might not seem like much right now, but keep these concepts in mind as you go through the course. Your next exercise is to take the samples you just generated from the multivariate normal and calculate the sample mean and sample covariance from your data. Consider whether or not these estimates are close to the values you expect as a bonus. Do not use the mean enco functions and numpty, but instead use the actual formulas. This will be useful for understanding many of the concepts in this course. All right, so I hope this little warm up has been pretty easy, by the way, you can see that this is not the kind, of course, where we’re going to spend two hours learning basic python, two hours learning pandas and so on. As you can tell from these warm up exercises, this knowledge is knowledge you should already have. It’s my general feeling that if you just learn basic spelling and grammar today, you won’t be ready to read Shakespeare tomorrow. And that’s what it would be like if you just learned basic Python and then tried to do financial engineering. Hopefully you agree with me that this is rational. So please take your time and remember that you have this course for a lifetime. There truly is no rush. And please note that I do teach no Penders Zippi matplotlib and so I could learn for free. Therefore, there is absolutely no excuse for not meeting these prerequisites. It’s easy, it’s free, and most importantly, it’s only beneficial to learn them. In other words, it helps you. So help yourself and give yourself the best possible start that you can by taking advantage of this free content.

10. Course Summary and Common Questions

1. Trading APIs and Deploying Your Strategy in the Real World

In this lecture, we’ll discuss the question that some students taking this course may ask, which is, how can I apply these concepts to a trading API? Some variants of this question may include how do I develop a financial trading bot to trade real money or how do I integrate the algorithms we discussed in a real time trading environment? Firstly, let’s recall one of my most famous rules, all data is the same. What does this mean? It means that no matter what your data is, the same algorithms will apply without any changes. My favorite example of this is to compare finance with biology. There’s no difference between a linear regression for finance and linear regression for biology. Linear regression. The model is the same in either case. From the point of view of your model, the data is just a table of numbers. It doesn’t care that your data is from finance or if it’s from biology. So why do I mention this? Well, one common beginner question is, what if I have data that is not exactly the same as what we used in this course, or what if my data comes from an API and not a CSFI? This should not be a problem for you, because as the rule states, all data is the same. So if your data is weekly, monthly or intra day data, this does not make any difference. If you’re using a different API or your data doesn’t live in a CSV formatted in the same way as it was in this course. This does not make a difference. You should be able to load in your data such that it conforms to the correct format. This is just the basic programming. Now, some students may ask about trading Epis is important to remember that students in this course come from all around the world and are very diverse. So it wouldn’t make sense to do something like show you how to use an API for US brokerage, which would require you to have an account in the U.S. if you don’t actually live there now, although it’s possible to get an account if you live outside the U.S.. Keep in mind that the laws are always changing. So one day your account may be fine and the next day it may not. Furthermore, it doesn’t make sense to show you the API for one brokerage when most students will not have an account at that brokerage. Also, keep in mind that for those of you with jobs in the field, everyone works for a different company. So the internal APIs at your workplace will not be the same as anywhere else. This also applies if you are looking for a job. There’s no sense in learning one API now and then having to change to a different API in the future. What companies are looking for is not the skill to use a single API, but they want you to have the skills such that you can learn any API at some companies, it might just be a matter of uploading a CSV and then having someone else or a computer program process your instructions. But again, it’s situation specific and it’s up to you to adapt to your particular situation. Another important fact to keep in mind is that things are always changing, APIs are created, APIs are destroyed in APIs are updated with breaking changes. One of my most famous rules related to this is learn the principles, not the syntax. If you’re just copying syntax that other people wrote, this is a sign that you don’t really understand what you are doing. And if that’s the case, then it would not be advisable for you to trade real money. This might sound harsh, but it’s really for your own safety. One great example of this is the quants open API. For years, this was a very popular platform for back testing trading strategies. Lucky for me, I decided not to include it in this cause. Almost exactly when this course was released, the quants opened, API was shut down. The lesson is do not depend on specific syntax and do not depend on specific APIs. Do be the kind of programmer that has enough scale to integrate with any API and use any syntax. Again, these are just basic programming skills. This has nothing to do with machine learning, data science or statistical analysis. It’s just basic coding. If you do not know enough basic coding to learn how to use a new API, then you should not risk putting your code into production. So essentially, the solution is this No. One get better at programming by improving your skills at programming, you will be able to understand APIs much better and understand how they fit in terms of how you can apply what you’ve learned in this course number to read the documentation for your API. Remember, this will be different depending on which country you live in, which brokerage you have an account with, or which company you work for. If you haven’t read the documentation for the specific API that you will be using, then there is nothing that anyone can teach you that will allow you to implement these concepts in the real world. If you cannot do these steps, then you should seriously evaluate whether it’s safe for you to be running any kind of algorithmic trading code. You should understand the code you are running. If you need to copy code from other people, then it strongly suggests that you are not ready to run your code and production.

2. High Frequency Trading (HFT)

So one question I get from students that I think is worth answering is this what about high frequency trading or HFT? Now, this is where a lot of beginners get confused. They think it’s just a matter of learning HFT. They think I’m gonna sign up for a course to learn HFT. I want to watch some videos on HFT, then I can do HFT in the real world. Now, this is going to be another one of those times where I have to tell you Santa Claus doesn’t exist. Here are some facts. No one, if you’re going to do HFT, you are not using Python, Python is a slow language. Remember, HFT stands for high frequency trading. Basically, you have to remember that these guys are so fast. There is some research papers that account for special relativity. If you don’t know about special relativity, the short answer is your basic assumptions about reality are wrong. When things move so fast that they approach the speed of light. And we know this stuff thanks to our old friend, Albert Einstein. Number two, you do not have the infrastructure to support HFT again, remember that HFT is all about speed. You cannot do HFT from your personal laptop. You need what are called co-located servers that are close to the exchanges. If you don’t know what this means, then you are not ready. Remember that there are billion dollar companies that install special fiber optic cables in the ground to improve their latency by milliseconds. So I may be making some assumptions here, but I’ll assume that you are not the type of student that can call up your local government and ask them for permission to install wires in the ground. And if you are, you probably wouldn’t need my help anyway. Number three, you don’t have the data and you cannot afford the data, as you recall, I mentioned earlier in this course that data costs money. The more specialized or hard to get your data is, the more it will cost. Now, HFT data will probably cost you thousands or tens of thousands of dollars per month granted if you have that kind of money to spend. I would be happy to consider you as a client. Now, there’s one scenario where you could be doing high frequency trading, which is this if you were an engineer that is already employed by a company doing HFT, then all this stuff is set up for you already. But there’s a contradiction here. You see, if you are already employed and your job is doing high frequency trading, then you would not be taking this course. If they hired you, then you should already know what you are doing. If that is not the case, well, then it’s your responsibility to figure out what the solution is. Now, you might think you can join this course if you want some insights for how to improve your business. But again, this is not the right fit for this course because I’m not serving as a consultant. If you want help with your business, I do have services for that. But an online course is not the right place. OK, so I hope this lecturer answered your questions about high frequency trading and how one might approach it in the real world. I hope that it helps you understand HFT in the context of this course and to address any concerns you may have had about why there wasn’t a section devoted to this interesting topic. Basically, the summary of this lecture is this. This is not a practical topic for this course. You don’t have the infrastructure, you don’t have the money. And we’re not even using the right programming language.

11. Extras

2. VIP Finance Enthusiasts, Beware of Marketers!

So I’ve said this elsewhere in the course, but I want to make it clear, for those of you who are really interested in finance, beware of marketers in order to help you be more cognizant of those who might be trying to trick you. I’m going to make you an offer. Anyone, even students who are not taking this course can message me at any time and ask me, am I taking a finance course taught by a marketer? Of course, I can only speak to the courses where I’ve seen the content myself, but the few that I have seen are ridiculously bad. Bad enough that it prompted me to make this lecture to warn you. Now, I won’t tell you which course it is or which instructor it is, but I will give you a yes or no answer telling you whether or not you are taking such a course. Typically, these courses claim to teach you things like predicting stock prices with storms and they will show you results like this with nearly perfect stock price predictions. Of course, those who are competent in finance know that this is nonsense. I’ll also include details about what mistakes to look for in these courses so you can verify for yourself why what they are teaching is dubious. I’ve heard that they are still making these mistakes and other possibly worse mistakes to this day. In other words, thousands and thousands of students have been learning wrong and incorrect material for years. Believe me, if you ever try to use something like this as part of your portfolio, when you’re trying to get a job, nobody will take you seriously. So that’s my offer to you. If you would like to know whether or not one of the courses you are taking is taught by a known a fake data scientist, just send me a message. You can do that either on this platform or you can go to my Web site at lezzy program or me either way is fine. Thanks for listening and I’ll see you in the next lecture.

12. Setting Up Your Environment FAQ

1. Anaconda Environment Setup

Everyone, and welcome back to this class. In this lecture, I’m going to go over a better way to install data science and machine learning libraries for Python, for Windows users, historically, Windows users have had a lot of problems installing this stuff. Luckily, these days there is an option that makes things very painless and just as easy as they are on Linux or Mac, that is Anakonda. In fact, even if you’re not on Windows, you can still use Anakonda. It’s nice because it isolates your environment from the defaults provided on your system. So for example, you can have Python three and Anakonda but Python two as your system default. When I first started these courses, I wasn’t keen on windows, since there were a few essential libraries that couldn’t be installed on windows without a significant amount of effort, if at all. In my view, anything beyond a couple of lines in the console or clicking in install file is too much and believe me, some students even have trouble with that. So it’s good not to make things too complicated before you can even begin the course. Nowadays, that has changed. It’s a lot easier to install things on Windows, in large part thanks to Anakonda. And so this lecture is all about how to install all the data science and machine learning libraries you’ll need on windows using Anakonda. So in this lecture, I’m going to walk you through how to install Anakonda as well as some of the libraries you might need that don’t already come with Anakonda. You’ll find that most of the common libraries, such as non-pay and zippi, are already included. So if that’s all you want to use, then for you it’s just a one click install. On this slide, I’m going to give you a super short summarized version of this lecture so you don’t have to walk through the installation with me if you don’t want to. For some people, that really helps since you can see it. But if you can do it on your own, feel free. So no one download and install Anakonda, this is just a one click install, it already includes Numpties Zippi Matplotlib and Pandas. That’s all we need for the numpties stack in python linear regression and logistic regression and a few more courses. It also comes with NHK, which is what we use for an LP Inside You Learn, which has some pre-built machine learning models. Now, even though this stuff comes by default, you can still update them if you want. So you can do QandA update number, for example, and that will update No. No, to install deep learning libraries, we’ve got PIP install tensor flow that’s going to install tens of flow. And if you want to install cameras, you have to first you can install PIP, which is going to update PIP, and then you can install cameras using PIP, install cameras. If you don’t update Pip first, you might get an error. Next, we have a synthetic, which is Microsoft’s deplaning library. So you do pip install and then the synthetic uro, which you can get from Microsoft’s website. So I’m not posting any URL here because the version could likely change in the future. And so you just Google search how to install synthetic and you can get a euro just like this. Next, we have pie talk, that’s Pip install my Nessy Peter Jayce, one, two, three, pie torch. After that, we have Viento, so that’s Condah and start Viento or QandA install V.A. pageview if you have an Invidia CPU and even start the cooler tool kit already. Number three, and open a gym that’s just Pippin’s gym, if you want to be able to play Atari games also, then that’s more involved. So just skip to the end of this lecture where I walk you through that. If you want to play and say videos using open air gym, then you also have to install feedbag. So the first thing we’re going to do is go over to the Anakonda website, that’s anakonda dot com slash download, scroll down to the Windows section and click on either Python three point six or Python two point seven. Or if you’re watching this lecture in the future and there’s a new version, get that the code in my courses is compatible with both Python two and Python three. So in that regard, it doesn’t really matter which one you get in the lectures, you might see Python to code. But the best way to make sure you’re seeing the latest version is to get Paul inside because repo. So make sure you’re always doing that, because I’m constantly making new updates. Now, though, Python three is newer, there are still reasons to use Python two. For example, in your work, you might use Python two or certain platforms like Google App Engine only support Python two. So if you’re running a Web app, that means you’re stuck with Python two. It does have great scalability features. So there are many good reasons to use Google App Engine. If you want to get more insight on whether to choose Python two or Python three. Just check out the appendix lecture. Python two versus Python three. So now that we’ve downloaded the install file, we need to do is click on it. That’s what I mean by one click install. You click on this here, OK, a few times and everything is done. Unfortunately, I gave my username Espace, which kind of sucks, but that’s what happened, so I’m sure some of you have a space in your username too. So if I come across any issues, at least you’ll know what to do. All right, so everything’s in stock. So essentially everything except the deep learning libraries have already been automatically installed. So you don’t need to manually install numpties, zippi MATPLOTLIB, Pendas, Epiphone or Saikat learn. So if you’re taking my number, of course, or any course that doesn’t use modern deep learning libraries, you already have everything you need. So let’s go into Epiphone and make sure that’s the case. To start a python, I go to the start menu type in Anaconda and then this Anaconda prompt application should pop up, actually it should pop up before you even finish typing anakonda. So we go into there and this brings up a command line terminal next, just type in Epiphone. After that, we can import all the libraries I mentioned earlier if we don’t get an error, that means they’ve been installed successfully. So let’s try something simple, like generating some random numbers and making a plot. So that’s a plot of random noise. Let’s make a histogram to. And so we see a normal curve just like we expect. And you can see that Tenzer flow is not installed, which is why we get this error, but we can install it very easily by exiting a python and then typing in PIP install tensor flow. Next, let’s try to install Carus. So we get this error, so I looked this up and determined that we need to update Pip, so let’s do that by typing in QandA install PIP. Conduct kind of works like Pépin that way they are all just tools for installing stuff. Now, let’s try Pip install cameras again. All right, so everything works. Next, let’s try to install NLG. This is used in my A.P. courses. So it looks like it’s already installed, so there’s nothing more to do. Just keep in mind, if we come across a library, you don’t care about it, feel free to ignore it. I find it’s useful just to install everything at the same time so that when you’re deep in the code later, you don’t have to think about stuff like this. So the next thing will install is synthetic. This is Microsoft’s deep learning, Liberi, notice how it’s not part of PIP. So you need to grab it yourself manually from Microsoft’s website. Unfortunately, it’s a little hard to find because there are many pages that deal with how to install synthetic on Microsoft’s website, but what you’re looking for is a page that has a link to a WHL file. So copy and paste that after Pippin’s. This is a good example because it shows you another way. You can use PIP by doing PIP install and then a euro. Next, let’s install Patrick. This requires us to use a custom source, so we have to specify the option minus C. Peter Gacy, one, two, three. So that’s kinda install minus C. Peter Gazy, one, two, three, PI talk. This is because a very nice guy called Peter Gazy one, two, three has provided us with a version of PI Torch that works on Windows. And once that’s all done, we can verify that they’ve been installed correctly by going back to a python. All right, so Tenzer flow works. And Altec works. See antique works. And torture works. Next, let’s stuff, yanno, Viento has historically been pretty difficult to install in windows, but nowadays that’s not the case. So if you go to their website, you’ll see a bunch of instructions. If you don’t want to use the GPU or you don’t have a GPU, then the instructions will be very easy. I don’t have a GPU on this machine, so I’m going to do the easy version. Keep in mind that Vienna is really great for learning purposes, so it’s totally fine even if you have a GPU to just install the C.P.U version for now and then use the GPU version of Tenzer for. Now, I ended up upgrading mail service and live python, since that’s what they told me to do on Vienna’s website, but it looked like these were already installed. In fact, updating mail service gave me an issue later. So we’ll have to fix that. So if you want to install Viento for a CPU only that’s kinda installed V.A., if you want to install V.A. for the GPU, then do QandA install theno GPU. Now, let me go into Epiphone and check if Sienna works. So we get an error because of this mail service thing that I mentioned earlier. So let’s set this environment variable it’s telling me to set. By the way, this is great to know if you don’t yet know how to set environment variables on windows, we can also check that it worked by using the Echo Command. So let’s try that again. Let’s do a simple example of adding two numbers in V.A. just to make sure everything’s working. If you want to do something even more complicated, you can run this script from deep learning part two, which doesn’t require any external data set. So just go over to the folder and in class two and type in python grid search Stoppie. So that’s going to look for hyper parameters using cross-validation. Now, in this last section of this lecture, we’re going to talk about reinforcement learning when we start studying reinforcement learning, there is yet another library we’ll need to install called open a gym. If you don’t plan on learning reinforcement learning, you can skip this part of the lecture. This has also historically been very difficult, but luckily the open source community has put in the work, so you don’t have to you’re welcome to read through the GitHub issue on this if you want. But I’m going to just do the simplest thing that works. So let’s first do Pippin’s Dolgen. Great, so that worked. Now, the second command is a bit longer, so let’s go to the actual GitHub issue and copy and paste the command. The easiest way to get there is just to go to Google and type in, install open Agim Windows Anakonda or something along those lines. So let’s paste that in. And notice how it feels, since I haven’t yet installed yet so we can install it by doing KONDA install yet. Now, let’s try it again. Unfortunately, this fails again because we need KCC, which is a C compiler. Now one way to get PCC is to do KONDA install MTW 64 toolchain. But unfortunately, I tried this and it also doesn’t work. In fact, I think this toolchain was installed already, so I tried quite a few things that didn’t work. So in order to save you some time, I’m going to recommend you only try this stuff on this page if everything else doesn’t work for you. So it works for me was to just grab the pre compiled binary directly. Now, in order to do that, you want to go to Coriolis, GitHub repo directly, so that’s this your URL here, GitHub, dotcom slash Karoly Attari dash pi slash releases. Next, you want to download the W.H file that matches your environment, so I have Python three point six on a 64 bit installation of windows. So this is the file I want. Luckily, we already discussed earlier in this lecture how to install a file that’s just pip install and then the path to the file. So let’s do that. Now we can test our installation by running a script that requires an Atari game, so let’s go over to our L2 and then Atari. And now let’s run DeQuan underscore tappy. Cool, so everything’s good. Now, the final thing we want to do for Open again is if you want to play a video or save a video, you want to install feedbag. So to do that, you want to type in a condo, install minus see Maypo, former Peg. Once you’ve done that, you can go to the Cardpool folder and type in Python, save a video pie, and this will run a script that will play the Cardpool game, show a plot and then save a video. And of course, you can also play this video by just navigating to the file and clicking on it. So for now, that’s everything. If I need to add new libraries or updates to this lecture, they will be appended at the end.

2. How to install Numpy, Scipy, Matplotlib, Pandas, IPython, Theano, and TensorFlow

Hey, guys, in this lecture, I’m going to show you how to set up your development environment so that you can follow along in these courses. First, let’s talk about operating systems, the three big ones are Windows, Linux and Mac. Most, but not all of my courses are part of the Deep Learning series. Eventually, the deep learning models were going to build will become so complex that we will need specialized libraries like Ciano intenser flow to implement them. Now, as of today, Viento intensive flow are not officially supported on windows. So if you are interested in deep learning, you will discover pretty soon that there is not much you can do without a ton of work. Even working with non GPU enabled libraries like numpty and matplotlib can be a challenge, but these are at least possible on windows. If you really must use windows and have your python environment inside windows and you don’t want to try the virtual machine method, which I’m going to discuss next. Then one good library I know of is called Anakonda. You can get Anakonda at Continuum IO downloads. I can’t vouch for this 100 percent because I haven’t used it myself, but I’ve known others who have found it at least usable. Now I’m telling you this from experience. I’ve worked with a ton of clients one on one, and python development, especially when it involves the numpties API stack is not easy when it’s on Windows. The method I’m going to describe using virtual machines will work on most modern computers. And as of right now, if you want to do any deep learning stuff, whether that be with piano or tensor flow, you cannot do this on windows. With that out of the way, I’m going to quickly go over which courses I’ve released so far or plan to release in the near future that do not require Viento intensive flow. So it’s possible to do on windows. With these in mind, you can decide if you want to install a virtual machine or not, but I would highly recommend it because it’s free and everyone can follow the same instructions. So we’ve got linear regression in Python, logistic regression in Python, which are the prerequisites to deep learning Part one, and we have deep learning in Python, part one, which is mostly in pie and a little bit of tensor flow. We’ve got easy natural language processing in Python data analytics. Ask you all for newbies, beginners and marketers, cluster analysis and unsupervised machine learning in Python and unsupervised machine learning, hidden Markov models in Python. This, of course, is going to have a little bit of yanno in it next. Here are some courses that depend heavily upon piano or tensor flow or both. We’ve got practical, deep learning in V.A. intensive, low, convolutional neural networks in Python, unsupervised, deep learning in Python and recurrent neural networks in Python. As you can see, a lot of interesting and complex stuff can be done in V.A. intensive flow. All right. So hopefully I’ve convinced you that using a virtual machine is a good idea. Windows is a great operating system, but it tends to not be as developer friendly since it tries to be more consumer friendly. Note that if you’re on a Mac, this probably isn’t necessary. You can just install Numpties Sipi Panda’s matplotlib, Viento and Tenzer flow using easy install PIP. Sometimes this stuff can fail, depending on the many combinations of versions of each thing that are possible, but usually just Googling your error message will lead you to the right solution on stack overflow. If you’re on a Mac, you can just do pseudo pip install, no Mississipi Python, Pendas, matplotlib and theno. Alternatively, you can use easy install, which might have a more recent version. You may need to use easy install to install PIP itself, which is just too easy and stop it. If you want to install Tenzer flow, just copy the command at Tenzer vlog. It’s just a PIP install command that points to their custom installation location. I won’t put it here since it corresponds to a specific version and that version may become out of date by the time you watch this. That is good news because the people working on Tenzer Flow are updating it all the time. All right. So if you’ve made it this far, that means you want to install a virtual machine with Linux on Windows or that you’re already using Linux and you want to know what commands to use to install these libraries. We are going to need two things to start for this tutorial. Virtual box and a lightweight version of Ubuntu. I would recommend Ubuntu or Elevon to I’m going to use the 64 bit version of Ubuntu for this tutorial. So download these first and then return to the tutorial. All right. So now that you’ve got virtual box installed, we’re going to create a new machine with 64 bit Ubuntu. So you want to hit new type in a name for your machine. And this stuff is already correct. I’m going to choose a. Two gigs of memory, you can choose more memory if your computer has more memory. I’m going to create a virtual hard disk now. I’m going to choose dynamically allocated and ECGs is good enough for this example. All right, so I’m going to go to settings if we go to storage. You can choose. The ISO file that you downloaded, so I’m going to load up the Ubuntu ISO that you downloaded from everyone to done, done. All right. So OK. And Head Start. So it’s going to take you through the installation of everyone to see you want to hit install. All right, so Element, who is going to take you through some prompts? All right, so you want to erase this, get install Ubuntu. And I’m not going to check any of these continue. All right, so once that’s done, it’s going to ask you to restart, so just restart now. All right, now that you’re in the machine, what I always like to do, if you notice how this window is too big for my Mac window, you can make the enter window sizable by installing just additions. And this also lets you do useful things like cut and paste between machines. So you want to open a terminal, you go to system tools, select ELEX terminal. Now I’m going to cry into. That folder. Pulls to the correct command was Ceruti slash Reebok’s Lennix editions that run. Notice how there was something that failed in there, so we need to install Jack. So you want to run Sudo and get up to. And then runs through app to get upgrade. So the next thing is you want to sue the app to get install, build essential. All right, once you’ve done that, we can try to run Reebok’s editions again so that sudo. Don’t slash. If he backs Linux. All right, so everything worked this time, so we’re going to restart this machine. All right, so now the window is only as big as I want to drag it. OK, so now we can install the actual data science stuff, so you want to open again elex terminal to a series of apps, get update again. And you want to install that it’s a long list, so Python non-pay, Python Sibai. Python, matplotlib, I, Python, Python, Pip, Python, Dev and Python set up tools. Yes. All right, so now that that’s installed, you can test it out, can type in Epiphone. So I python is open, import, vampy. Implies working in Port Sibai Import Panda’s. Right, we don’t have pandas yet. We will install that next import matplotlib. So we have everything we’ve installed so far. So now we’re going to install pandas and piano, so that’s Pippin’s a minus minus upgrade pandas the. All right, so now we’ve got Fiorino, the last thing we need is Tenzer flow. To get this, you guys want to go to the Tenzer Flow website and just grab the latest command since they’re updating the package all the time. So let’s open a browser. Let’s search for and Tenzer flow. All right, you take this code. Copy it, you paste it in here. All right, now you’ve got V.A. intensive flow, you can test them out, go to GitHub, dot com slash lazy programmer slash machine learning examples. But. All right, so you need to get if you want to check out this code with get, so let’s install get. All right, so let’s run our clone command again. Let’s try the HTTPS version. OK, so this one works. All right, so you see into machine learning examples go into an in class, too. So this has some introductory. Tenzer, flaunt the anecdote, so just run Python V.A. one up high, since this does not require any data. Cool. So it seems everything is working now let’s try to Tenzer, for example, so it’s Tenzer flow one. All right, so you want to make sure you install the correct Tenzer flow, you want the CPU only version. Hopefully it just overrides the other version without any trouble. So let’s try the example again. Super. So now everything is working. Now, if you want, there is a text editor that I highly recommend, it’s called Sublime Text. So you had download. You go to Bunta 64 bit. You save this. And it will automatically be installed. All right, so now you have the exact same text editor that I use in my lectures. So you open machine learning examples, and this is all of our code.

13. Extra Help With Python Coding for Beginners FAQ

1. How to Code by Yourself (part 1)

Everyone, and welcome back to this class. This lecture is about how to write code independently when you’re doing machine learning. You always hear me say that the first thing you should try to do when you learn an algorithm is to go and write it up and code by yourself before looking at someone else’s code. For some students who are new to machine learning, it’s not quite clear how to do that. So this lecture is going to explain the thought process you should go through in hopes that it encourages more people to try to code by themselves. I like to encapsulate this in the phrase, When it’s time to code, you must code. So remember this whenever you see some code that demonstrates the concepts we learned in this course, even if you don’t fully understand what’s going on, coding and copying examples from others helps you build muscle memory. Sometimes people act like they are going to learn tennis by reading a book about tennis. This is, of course, obviously not possible. You need to learn some things about tennis, which are pure fact and reside in your brain as conscious facts. But then you have to go out and practice those concepts on the tennis court. Once you’ve exhausted and mastered everything you know so far, then you can learn new techniques. So you have this continuous cycle of learning new techniques and then practicing them. Why is this because learning the technique as pure fact doesn’t mean you truly understand it, practicing the technique helps you think about the technique from a different perspective. Practicing again and again leads to new understanding, to learning in the subconscious, which is muscle memory. You must go through the cycle. But some students naively think they are going to read an entire book about tennis and then be a tennis master the first time they try playing. This is very common, especially when all you really need to do is sit there and watch a video and no one can force you to type. A lot of people don’t even think to try. So just remember this. Whenever you see code, when it’s time to code, you must code. First, we are going to talk a little bit about why you want to code buy yourself a lot of the time what an algorithm does or how it works isn’t quite clear. The first time you learn about it, every individual will have their own unique background. So they might be familiar with the patterns and know exactly what to do while some individuals might have questions that no one else has. Sometimes you gloss over details because you assume you know what’s going on when really there are important things you’re not considering. So this is how trying to code by yourself can help you. Coding by yourself forces you to think about each and every detail. It forces you to think line by line. You have to be familiar with the data types and the shapes of all your variables, and they all have to fit together properly, kind of like Lego blocks. As an example, you know that in order to do element wise matrix additions, both matrices have to have the exact same shape. So if you find that they are not the same shape, then one of your previous assumptions was incorrect. So you should go back and correct it. Lego blocks have to follow a specific set of rules in order to fit together properly. If you make incorrect assumptions about how Lego blocks fit together and you try to join them. It’s not going to work without trying to build things by yourself. You will never discover these details. Let’s now talk about how to code by yourself. Consider the supervised machine learning scenario. We know that we’re going to have some inputs and some targets, and we want to try to make predictions from the inputs that are very close to the targets. We typically call the inputs X and the targets. Why? Sometimes we call the targets T, but for the purpose of this lecture, we’ll call the targets Y. Now, this next point is a key point. It doesn’t actually matter what X is and what why is it does it matter if you’re looking at an e-commerce data set and the columns of X might be time on site, time of day, how many pages the user looked at and so on. You could just as well be looking at a data set of x ray images where each column is the pixel intensity of an image. This is the key point. We say all data is the same when we do linear regression. We don’t have different kinds of linear regression for e commerce and x ray images. Linear regression is the same algorithm no matter what your data set is. So we say all data is the same. Theoretically, I could give you a data set of X’s and whys and you could train a classification algorithm on it without me even telling you what the Xs and Ys mean. You should get very comfortable with this idea. Sometimes people say, well, this isn’t practical because I want to do concrete examples, but that’s because they’re not thinking intelligently. In fact, this is the most practical thing we can do because it means everything we learn. We can apply it to any dataset that exists. It means that I can train a model on an e-commerce data set, but I can also train a model on an online advertising data set without learning anything new at all. It’s truly the greatest form of lazy programming. Learn something once and apply it to every industry. One great consequence of all data is the same is that there is an unlimited amount of practice opportunity for you. You can download data sets from Kaggle, from Google, from Wikipedia, from Amazon or from anywhere else and try the algorithms you learned. What this means is let’s say you have a data set you care about more than something we do in Class C in class. We have to use data sets that everybody can understand, for example, text and images. Everybody should know what text and images are. Text and images are just fundamental data types on the web. You probably don’t even think about the fact that text and images are data because it’s so trivial. In any case, we work with text and images a lot because everybody understands them. But let’s say you’re a biologist and you know about DNA. You find DNA and genomics really interesting. So you want to use the algorithms we learn on that. Well, that’s great. And you should totally do it. Remember, all data is the same. So all you need to do is convert your data into the appropriate Xs and wise to plug into our machine learning algorithms. The downside is we most likely won’t talk about DNA in class unless it’s a very simple example because most computer scientists are not biologists too. So they don’t know what DNA is. They don’t understand the specifics. So an example wouldn’t make much sense to them. This is opposed to text and images, which makes sense to everybody. Same goes with any other specialized field like finance, computer networking, cosmology and so on. So now we can start off from the right place to talk about supervised learning, we have our exes and we have our wise, what do we do with them in supervised learning? We know that there are two main things we want to do. We want to do training and we want to do prediction and psyche to learn. All the models have the same API. It doesn’t matter if you’re doing logistic regression or decision trees or random forest unsupervised models in Saikat learn to have the same two functions fit and predict. Fitting a model is just a synonym for training a model. If you don’t believe me, you can go look at the psychic documentation yourself. When you’re learning about supervised machine learning, what you’re really learning is what goes inside these two functions, what are the parameters and how are those parameters learned? That’s really what I’ll supervise. Machine learning is filling in these two functions. Taking this perspective is going to give your code structure and it’s going to make things much easier to visualize in your head. Let’s now do a simple example to solidify this idea and this example, I’m going to give you an algorithm and you will implement it in code. There are two key points I want to make before I give you the algorithm. First, I’m not going to give you much intuition about how it works. Second, I’m not going to derive the theory behind why it works. The reason I want to mention these two key points is that you should realize you don’t need these two pieces of information in order to translate pseudocode into code. A lot of time, these three concepts, intuition, theory and implementation work to reinforce each other. The reason I’m focusing on implementation right now is because it’s the one people most often miss or think they don’t need at all, when really it’s an extremely important part of the learning process. OK, so the pseudocode is as follows in my predict function, as you already know, I’m going to take in some input data X my predictions will be Y hat equals X times W. You might already recognize this as linear regression. However, forget what you know about linear regression and pretend that this is just some formula I gave you. What I can tell you is that we are creating some kind of regression model. It should be clear from this equation that the parameters of the model are contained in this vector of weights. W I would probably also tell you that W is a vector that’s the same size as the number of feature vectors in X. Of course this must be true in order for the Matrix multiplied to be valid, this axis kind of a sanity check so you can make sure everything is making sense. In my fit function, I want to perform this loop some number of times, since we know from the previous slide that the parameters of the model are contained and w then it should be clear that inside the fit function, what we want to do is update W. Not surprisingly, that’s what’s happening here. Another good sanity check now, because you’re already familiar with linear regression, you probably already recognize this as gradient descent. Again, I want you to pretend that you don’t know that, but what do you know? You know, that iterative of algorithms are common in machine learning, and that gradient descent is among the most common. You could probably infer that this is some form of gradient descent, but you don’t need to know what it is in order to write it in code. What else do we know? You know what a for loop is and you know how to write one in Python, you know that axis and by the Matrix containing the input data and that Y is an inside vector containing the targets. You know, that white hat is an inside vector containing the predictions, you know, that W is a disease vector containing the model parameters and you know how to add, subtract and multiply these objects so we can write this algorithm without even knowing how it works. Of course, a lot about how it works can just be inferred from this formula. Here is what you don’t know. You don’t know T- the number of times the loop is supposed to run, you don’t know Ayda, the learning rate. This is completely fine. You can’t let this lack of knowledge stop you in your tracks. The truth of the matter is there are going to be situations where you’re not told the exact numbers to plug in. In many cases, the answer is going to be it depends on the problem. The important thing is you have to get used to the idea of trial and error and you have to make educated guesses based on what you already know. So what can you do when you know that the learning rate is supposed to be a small number, how small? It depends on the problem. Typically, it’s a number less than one like zero point one. If that doesn’t work, you might try lowering it. Typically, we lower it by factors of 10. So, for example, the next learning rate we would try is ten to the minus two tenths of the minus three and so on. Typically in machine learning, whether you’re looking at a supervised or unsupervised algorithm, there is an associated cost or objective function that you’re trying to minimize. Let’s suppose I tell you it’s squarer, which is typically what we would use for regression. Well, now you have a way to choose the number of iterations of the loop and the learning rate. How do we do this? So inside your fit function, you would plot the cost as a function of iteration, I always recommend doing this. No matter what algorithm you’re looking at in general, you always want the cost to converge. The pattern you typically see is that there is a steep drop at the beginning and it slowly flattens out as the number of iterations increases to choose the number of iterations you want to stop. When the curve is sufficiently flat, you might recognize this as a diminishing returns scenario. The returns diminish because at the beginning, we reduce the cost by a lot in just a few iterations at the end we can do a lot of iterations, but we reduce the costs by almost nothing. An alternative is you could use a separate validation set and stop the validation cost increases, but that is not the focus of this lecture. Unlike the training cost, the validation cost is not guaranteed to decrease at every round because that is not what we are minimizing with respect to. You can also use the same plot to help improve your learning rate if the cost explodes or becomes not a number, your learning rate is probably too high if your cost converges to slowly. You can try increasing your learning rate. So what does your final code look like? Well, we know we want to create a class with at least that predict and fit functions, you don’t have to use the class, but I find that it encapsulates the code nicely and provides useful structure. Notice that once you have the algorithm in math, there isn’t much work to convert it into numbered code. You know how to do matrix multiplication. You know how to add. You know how to subtract. These are basic arithmetic operations that I’m sure you know already. If you don’t know how to do these in Mumbai, then I have a completely free number because on the number I stack that you can take. Here is the second part of the code where we actually create an instance of the class and then use it. I’ve also included the squared error cost function just for completion sake. What’s cool about this template is that it doesn’t really change from one algorithm to the next. The structure is basically always going to be this way, at least for supervised learning and unsupervised learning. So as we discussed before, it doesn’t matter what algorithm you’re implementing, whether it be linear regression, logistic regression, a neural network and so on, they always have the predicate function and the fit function and learning what goes inside these functions is equivalent to learning the algorithm. So, in fact, one way to begin coding is to start with just this boilerplate code and then fill in the blanks.

2. How to Code by Yourself (part 2)

This is what is known in software engineering as test driven development, or TTD in test driven development. We start by running tests because, number one, your code should always have tests. And number two, running the test first forces you to think about how you expect your API to work, and it allows you to make sensible design decisions without being distracted by implementation details. So you see how this applies to every machine learning algorithm, not just linear regression. And a lot of my older coding videos, I unfortunately decided to just write the code straight through this helps some people, but it’s not ideal for others. In addition, the coding videos will appear without warning. Generally, the lectures appear in the same pattern. First we do theory and then we do the corresponding code. It pretty much always works this way. So after the theory lecture, you can be pretty sure we are going to implement that theory in code soon, unless it’s so simple that it can be an exercise now to avoid treating everyone like they’re in kindergarten. I don’t have videos telling you that it’s time to code. You’re all proactive and independent adults. So when you learn some theory, you already know that it’s time to try to put that into code by yourself. Sometimes you might get stuck and have no idea what’s a writing code or what the data set is, in that case, it’s fine to watch the next coding lecture, but don’t watch it all the way through. Watch just enough so that you know what the general idea is and what we’re trying to accomplish and then try to finish the rest of yourself. Now, let’s say you’ve watched just enough so that you know what we’re trying to do, you know the theory, but you still can’t complete the code at this point, you may want to revisit the lecture theory to make sure you really understood each part. Failing that, you may want to try and get clarification on the discussion board. Failing that, you may want to look at external resources as well. Sometimes you might be missing some important computer science background that I’ve assumed everyone already knows. That’s perfectly fine because everyone has a different background. It’s highly encouraged to do your own research. And this is also perfectly normal. It’s not a bad thing. It’s a good thing. Ask any university student if this is what they do and they’ll tell you that they do it all the time. This builds a level of mental toughness and initiative taking. That’s important. If you want to learn this stuff, there is a lot to learn and I myself am still learning new things every day. So let’s say you’ve watched the theory lecture, you’ve looked at some external resources on the theory, and you still don’t understand how you would put it into code. Well, at this point, there are some great exercises you can do. Watch the coding lecture or look at the focus on GitHub, go through it and make sure you understand each line. The next step is without looking at the code again, try to implement it yourself. This by itself is a great learning tool. Many of the algorithms I’ve learned, I didn’t understand them the first time I learned them by looking at someone else’s implementation and dissecting it. I looked at each part and I asked myself, which parts did I get right and which parts did I get wrong? This helps you understand the mistakes you were making previously. The cool thing about implementation is it works backwards to we go from theory to code, but thinking in the reverse direction, the code actually helps us understand the theory better. Also, I always say that if you can’t implement the code, then you don’t understand the theory. So if you can learn the theory 50 percent of the way, try to implement the code, look at an existing implementation and then finally get it working. Then you can be guaranteed that you now know more than just 50 percent of the theory. The active coding helps you understand more of the theory than if you had just learned the theory by itself. So implementation helps you understand the theory, but it also helps you understand the intuition as well. Intuition is a tricky thing. In fact, I would go so far as to say it’s a trap. In many or maybe even most cases, intuition is built on experience. That means driving the theory from first principles or in other words, driving the theory from facts you already know to be true and from implementation. Students often want to hear intuition from people with lots of experience, but the problem is that this intuition is built on hindsight. Intuition is often both the first thing and the last thing you want to focus on, you want to have a little bit of intuition at the beginning so you have some general idea of what you want to accomplish. Then after you learn the theory and implementation, you can reflect on the details and summarize them at a high level. This is real intuition built on hindsight and experience. The big mistake beginners make is that they read some intuition written by someone else who actually has experience when they themselves do not. From an outsider’s perspective, it’s hard to tell the difference. Why is that? Well, if the beginner can regurgitate the same high level ideas as the expert, then it would sound like they are also an expert when in fact they are not. There are unfortunately, too many courses out there that do this, they want you to feel good that you learn something and they want to avoid you having to do anything too difficult. And because you can repeat the same high level intuitions, it actually sounds like you’re an expert. So this is why I call it a trap. Knowing intuition not only makes you feel like you know something, it also makes you really confident about this fact. And then because your ego wants to maintain this confidence, you’ll start making crazy statements like the theory and implementation are not important because you already understand the intuition. But as you know, this is completely backwards. Let’s summarize this lecture, since that was a lot of information to take in, remember, this lecture is all about how and why you should code by yourself. First, we talked about why this is because it ensures you are thinking about the details and filling in the blanks using your own pattern recognition skills. It ensures you don’t just assume you know what’s going on, but that you actually know what’s going on. There’s a big difference, especially if you’re the type of person that likes to assume you know everything. Next, we talked about how we talked about how all data is the same. And it doesn’t really matter what domain or industry your data is from. The same algorithms all apply. Next, we looked at the interfaces to the algorithms, which are also conveniently all the same. So we have a kind of mix and match type of scenario where we can input any kind of data into any kind of model. For example, you can use linear regression with financial data or you can use a neural network on ecological data, all data is the same and all models have the same interface. Next, we talked about a strategy for how to actually apply this, you’ve probably already seen that we always alternate between theory and code. First we discuss the theory and then we implement that in code. Sometimes you might have to go back and revisit the theory or sometimes you might have to look things up on the Internet. That’s perfectly normal. This is pretty much the day in the life of a university student. We’re trying to build the same level of mental toughness and initiative taking in this course. So the strategy is stop after you learn the theory and try to code yourself. Sometimes you need a little more detail, so you might want to watch a little bit of the coding lectures to see what’s going on and then try to implement the rest of yourself. If you can’t get it by yourself, then maybe try watching the whole thing or looking at the expected result and then trying to implement it from memory. This by itself is a great learning tool because it allows you to ask yourself, what did I get right and what did I get wrong? So you can understand the mistakes you were making. Finally, it’s important to understand the difference between intuition, theory and implementation and how they reinforce each other. It’s important to not fall into the trap of only understanding intuition written by someone else, and then assuming that this equates to actually understanding everything in many cases, it’s not about waiting for someone to tell you some nice phrases that trick you into thinking. You understand the intuition, but rather you should seek intuition by working through the theory and the implementation. And lastly, don’t forget, when it’s time to code, you must code.

3. Proof that using Jupyter Notebook is the same as not using it

In this lecture, I’m going to talk about Jupiter notebook. Occasionally I get this strange question from students asking why I don’t do things in Jupiter notebook. Let me explain why I think this is strange. Firstly, it’s my position that it’s completely unnecessary and actually doesn’t change anything to use Jupiter notebook or not. Let me repeat that. Using Jupiter notebook is exactly the same as not using Jupiter notebook. There’s no difference other than the fact that it looks different. In other words, the screen is a different color. Obviously such a difference is trivial in this lecture. I’m going to demonstrate how that’s the case. One major reason I dislike Jupiter notebook is because it causes too many students to be unaware of how Python really works. If you’re only comfortable inside notebook and when you see Python code in a text file or anywhere else and you get scared or intimidated, that’s not good. That python code that’s in the text file is exactly the same as what would be in the notebook. There’s really nothing scary about it. Programmers working at real jobs eventually need to write code that’s going to be deployed and run automatically. In other words, your final code is going to sit in a python file that runs by itself without you open a notebook. So if you’re going to have any hope of using these skills in a real job, you had better be comfortable with writing Python code outside of Jupiter notebook. You would also better be aware that there’s actually zero difference between writing code in a group and a notebook and writing code in Python or the console. Here’s one example I like of how you might use Python in the quote unquote real world. Let’s say you write a script that emails your boss to tell him you’re going to be late for work. And let’s say you don’t actually want to send this email manually, but you want it to get sent automatically every Friday morning so that your boss doesn’t yell at you for coming in to work late after you party too hard Thursday night. Well, that’s very simple. All I have to do is on my server, create what’s called a crosstab in it. I just enter the code for when I want this command to run. And then to the right of that I specify the command that I want to run. That’s just python space. And then the script name, as you can see, it’s simply how you would run this Python script from the command line. And now every Friday at nine forty five a.m., this script is going to send the same email to your boss. Tell him you’ll be late. Well, let’s not get off track here, the point of this is you really don’t want to be using Jupiter notebook for something like this. I think one perceived advantage of Jupiter notebook is that you can see the results of intermediate calculations. However, this is merely a perceived advantage and not a real advantage because you can do the exact same thing even when you’re not inside notebook. Firstly, as I’m sure you’ve seen by now, I Python also print out the results after you enter a command, ie Python is called a Reppel, which stands for read eval print loop. And that’s generally what they all do, no matter what language you’re in. So whether that’s Python, Ruby, P or any other language, the keyword here is print. Why is that? Well, one of my golden rules for writing and debugging code is when in doubt, print it out. I can’t tell you how many times I’ve gotten a question on the Q&A when it could have been easily answered by inserting a print statement into the existing code. Anyway, what’s the point of this long discussion about printing things out? Well, it’s that if you think Jupiter Notebook is the only program that helps you do this, you need to expand your horizons a little bit. You should, in fact, always be doing this. If you’re not using an abundant amount of print statements while coding, you are not doing it right. Remember, programming is not philosophy. You’re not supposed to be running a program in your head that’s like trying to do long division in your head when you have a calculator in your hand. So just by printing things out, you can be more efficient. Stop trying to guess what a program will do and just let the program itself tell you what it’s doing. The key takeaway here is you should always be printing things out. The fact that Jupiter notebook shows you the result of each block of code is not simply a happy surprise. But another important lesson here is that if you want to use Jupiter notebook, there’s absolutely nothing stopping you from doing so. In other words, using Jupiter notebook is 100 percent compatible with everything we are already doing. In fact, if you recall, your goal in these courses is not to run my code, but to write your own code. And of course, since it’s your code, you can write it however you want, including Jupiter Notebook and the rest of his lecture. I’m going to prove to you that you can take any script from our course repository, which we know runs in the console, because that’s how I always demonstrate it and show you that this exact same code runs in Jupiter notebook. Let’s begin. OK, so let’s say I am in the folder Numpty class, and I’m interested in the code inside classification example that PI, as you can see, what I have right now is this code inside a text editor. Now, if you’re not aware of what a text editor is, it’s just a program that shows you the contents of a text file and lets you edit those contents. It’s the ideal program for writing code. Now, occasionally, if you’re writing in a language like Java or Swift, you might want to use an I.D., but even then, it’s totally optional. These days I prefer to write Java in a plain text editor like Sublime Text as well. In any case, normally one does not need to use an idea for writing Python code now about using a notebook. Well, let’s start up a Jupiter notebook. So I’m going to go Jupiter notebook. All right, so now I’ve got Jupiter, a notebook running, so I’m going to start a new notebook. Also, notice that we started the notebook in the same directory as the relevant Python file, so that’s something you want to keep in mind for the future. Now, at this point, what I’m going to do is I’m going to prove that everything in this Python file works exactly the same in the notebook as it does in the console. So let’s start with the imports. Let’s grab this one to. All right, paste that in. Run it so everything’s fine so far. Now let’s load in the data. OK, we’ve loaded in the data now, I’m not sure what its type is, so I can check it by using the type function. So let me try that. Cool, so I get Escala and utils, that bunch, so that’s the type of the variable data. Now, previously, you recall that we did this example in Python, but as you can see, the result is the same in Jupiter notebook. Alternatively, if you wanted to run this Python file in the console, so you wanted to type. Let’s say. So let’s say you wanted to type in python classification example, that pie, then you could just add some print statements if you wanted to show those same lines while this file was running. Now, I’m not going to go through such elaborate detail for the rest of this example, since you’ve already seen it, so let’s just get through the rest. Line by line. Let’s say I want to check out the keys in the data variable. OK, looks good so far. Let’s check the shape of the data attribute. OK, looks good so far, check the targets. OK, still, what we expect the target names looks good. Target shape, it should be 569, yeah. And let’s check out the feature names. OK, that’s the same as well. Now let’s do our train to split. OK. Now let’s instantiate and fit our model. All right, now let’s check the train score. And the test score. All right. Now let’s see how we can make new predictions. How have you assigned to a variable, it doesn’t print the result. But if you have an expression, then it does print the result. So those are some predictions, and this was an alternative way to calculate the accuracy of the predictions, so we get the same answer as before. And we also have this other example where we can use a neural network to do the same thing. So let’s build the model, train it and print the train score. And let’s print the test score as well. All right, so everything works the exact same way as it does without using notebook. Now, another thing I could have done was I could have just taken this whole thing and copied and pasted it in and run it, so that’s another possible thing you can do. But the downside of that is you don’t get to see the intermediate outputs. But again, that’s what print statements are for. All right, so what can we conclude from this exercise? Well, we can see that this code runs exactly the same inside group in a notebook as it does everywhere else. That’s why I always say python code is python code, no matter where it is. If you want to use you put a notebook to run the course code. There’s absolutely nothing stopping you from doing so.

14. Effective Learning Strategies for Machine Learning FAQ

1. How to Succeed in this Course (Long Version)

Everyone, and welcome back to this class. This is a short lecture I like to give and each of my courses on how to succeed in this course, this is very important to talk about because you’re learning a difficult technical subject. It’s easy to get stuck or even worse, give up because you’re just not getting things. Maybe there are too many gaps in your knowledge so you can’t connect the dots. Maybe some assumptions you’ve made is incorrect. And keep in mind, there’s a lot of material in this course. It can be overwhelming if you’re new to this field of study and you’re not familiar with the conventions and the typical patterns we use. Now, if you’ve already seen this lecture in the past, you are more than welcome to skip it if you remember what’s in it and you’re already using these guidelines. The first guideline is this, remember that I want you to succeed. In fact, that’s why I’m giving you these guidelines in the first place. So if at any point when you’re watching these lectures and you become confused, just ask on the Q&A, also known as the discussion forum. Typically, I respond within hours, sometimes even within minutes, if I happen to be on the site and I see a notification, I answer any and all questions. So even if you think it’s stupid, just ask anyway. One example I get from time to time is where do I get the code for this course? And of course, my answer is typically, well, it’s in the lecture called Where to get the Code for This Course. At the end of the day, no one cares if your question was trivial or silly. If it’s something keeping you from progressing in the course, I want to answer it. Now, very rarely I get people saying, well, I was so confused that I couldn’t even think of the right question to ask, and this is where you have to take some responsibility for your own learning. You have to ask yourself, what did the instructor assume? I know that I didn’t actually know. Maybe I can look into those things and that will help me formulate a question. Maybe there’s some background knowledge I’m missing. A lot of people like to have exercises to practice when they’re taking a course. The active thinking of the right question to ask is itself an exercise. It might not be the type of exercise you thought you’d be doing, but in many ways this is more profound. This is an exercise that requires you to think and to change how you think. And ultimately that is the best value you can get out of a course. Speaking of background knowledge, this is one of the most important guidelines in this lecture. You may have noticed that in the course description, I’ve listed some prerequisites or in other words, things you should know before starting this course. You might wonder why do I ask you to know these things, why don’t I just teach them in the course? And while we do review some topics in this course, it’s not the focus. There’s such a wide variety of students in this course, students from almost every country in the world. Students of all ages. Students of all backgrounds. What does this mean? That means let’s review for one person is going to be totally new for another, some people are going to tell me they don’t know calculus. Some people are going to be very far behind with their python programming skills. But if we start teaching Python programming skills, other people are going to say, hey, why are we doing Python? Isn’t the topic of this course deep learning? And I tend to agree that we should stay on topic. So how can we reconcile this? Well, it doesn’t make sense for me to give you an entire computer science education in this one course, because a lot of that would be irrelevant for a lot of people. The only logical thing to do is for all of us to meet at a common meeting point. How is that point defined? Well, it’s defined by the core prerequisites. In fact, they are so important that in certain instances I’ve listed the prerequisites twice, just in case you missed it the first time. And I’m also reminding you now for good measure. Now, one pitfall is that people sometimes assume they know things that they don’t really know. So if you took calculus 20 years ago and you got a C and you skipped class and the times that you did show up for class, you were sleeping well, probably you don’t know that much calculus. But remember, I want you to succeed. I want you to be able to learn this material. I can point you to resources where you can pick up this missing background knowledge. But the key is you have to ask. In some cases, there are certain places where I’m not allowed to post certain links, so we have to be mindful of the rules of the platform we’re using. Now, occasionally people ask me, well, why don’t you just teach a course for pure beginners, why don’t you teach these courses in such a way that they don’t require lots of math and lots of prerequisites? Well, I don’t want to turn this into an easy beginners type, of course, because for one, that’s not what I’m interested in personally. And number two, don’t forget that this is state of the art stuff. State of the art generally isn’t easy. And if it were easy, that would mean everyone is doing it. And if a thousand other people can do the same thing as you, well, what’s a potential employer going to think? What are your chances of getting that job when all 1000 of you have the exact same skills? So we don’t want skills that are easy to get, we want skills that are hard to get, skills that put you at the top of your game, skills that not many people have. If I’m hiring someone, I want the guy who stands out, not the guy who says the same thing as everyone else. Finally, the final guideline for how to succeed in this course, I mentioned earlier that there’s a lot of material, so much that it can be hard to keep track of. In addition to taking your own notes and doing all the derivations yourself, how can you be absolutely sure you’re understanding this material? I would, in fact, consider this even more important than the derivations. Not everyone is going to be a pro at math. So it’s understandable if you want to skip a few steps, sometimes it’s possible to compartmentalize some concepts or delegate learning them for some other time, but still understanding the high level picture and how it fits in with everything else. What you cannot skip is the implementation. Implementing these algorithms in code is the real test for whether or not you understand the material. And this is not about reinventing the wheel or that you shouldn’t roll your own whatever, because someone else out there can do it better than you, that is a software engineering principle, not a principle of learning in production. You don’t want to write your own hash map or your own string class. And yet what do we usually do as an exercise in a typical intro to computer science class? Well, we do precisely that. We implement hash maps and we implement string’s. And why do we do this, because we want to have a better understanding of how these things work, if you make an assumption that’s incorrect or your code is probably just not going to work, merely using a hash map and using a string only gives you a very superficial understanding. That’s beginner stuff, that stuff everyone knows. That’s not stuff that’s going to help you stand out from the competition. And it’s true that you’ll never use your own homemade hash map or your own homemade string in any code you write. Unless, of course, you are a professional that focuses on that sort of thing, they’ll probably be inefficient and they’ll probably have bugs. But that’s OK because you’re not doing it to make it perfect. You’re doing it to learn. You’ll notice that the general pattern that this, of course, follows is that we always discuss theory and then we implement that theory in code. So we’re always going from theory to code, implementing everything that we learned. If you can try to implement something before watching the coding lecture and the appendix, I have a lecture called How to Code by Yourself that goes much more in-depth into this topic, along with multiple strategies for how to approach implementation exercises. So to summarize, these are the guidelines I think are going to help you succeed in this course, number one, make generous use of the Q&A even when you can’t think of the right question to ask. Number two, make sure you meet the prerequisites to this course if you’re not sure what they are or you’re not sure how C. guideline number one. Number three, try your best to implement everything we learn. This is the true test for whether or not you understand something. Remember that it’s perfectly normal to get stuck once in a while, I get stuck all the time when I’m learning, that doesn’t mean I’m not smart, nor does it mean the book or video I’m learning from is bad. It means that there’s a disconnect between what the author believes I knew and what I actually know. And as a learner, I do my best to close that gap. A good skill to have is learning how to compartmentalize concepts, so here are the things I understand, here are the things I don’t understand, but here are the things I understand about the things I don’t understand. And these are the holes. So you know what you’re missing and you know where they fit. And you can work to fill in those gaps over time. One thing I’ve been doing quite a bit recently is adding review lectures to my courses. These aren’t meant to teach you everything from scratch, since that’s going to take you a long time. But they are meant to help you recall certain things you may have forgotten. So if you have an idea for a review lecture that you like to see added to this course, please feel free to make a suggestion using the Q&A.

2. Is this for Beginners or Experts Academic or Practical Fast or slow-paced

Everyone, and welcome back to this class. This lecture is designed to answer some common questions I get about how difficult this course is and where it fits in terms of academia, the professional world and its practical applicability. So rather than answering this question individually over and over again, it seems like a better idea to make a lecture about it and answer everybody. At the same time, a lot of people become confused because this is their first machine learning course and they just don’t know where it fits in the grand scheme of things. So is this course academic or practical and is it for beginners or experts? Is it fast paced or slow paced? I’m going to answer the second question first, is this cause for beginners or experts? Interestingly, the problem with this question is a problem for machine learning to let me explain why this question is itself problematic. One of the challenges in natural language processing is that of ambiguity, when someone says some words, the problem is those words can mean multiple things and it just so happens that I have some courses on it up. So this is a problem I think about often. So how does that apply here? Well, when we use the words beginner and expert, we have to ask a beginner from whose perspective an expert from whose perspective for some of you, you might think that a beginner is a random person. You take off the street who knows nothing. But even that is ambiguous. There are people out there who do not even really know how to use a computer beyond turning it on and checking their email and browsing the web. There are people out there who are pros at Microsoft Office so they know how to use a computer and make a Microsoft Word document or a spreadsheet, but they’ve never done any programming before. There are people out there who are very good developers and have lots of programming experience, but they have no idea what machine learning is. So all of these people are beginners in some way, but each of them would require a vastly different amount of work to prepare for this course. It’s the same situation with experts most people consider anyone who knows more than them to be an expert, but of course, how much you know and how much some other person knows is relative. If you know how to plug and play into psych, you learn and nothing else, you might seem like an expert to someone who only knows Excel, but you’ll seem like a beginner to someone who’s read Bishop’s pattern recognition and machine learning book. So just to finalize our answer to this question, is this cause for beginners or experts? This question is misinformed to begin with because beginners and experts are not well-defined terms. Instead, we have to go back to what is the specific skill set you need before starting this course. And of course, that’s defined by the prerequisites which are listed in the course description. So precision is the key here. We want to know what is the precise list of skills that I need before taking this course. This is much better than ambiguous terms like beginner and expert. Another matter is that people are going to measure their own skills differently. That’s a much tougher issue to deal with. Sure, I can say you need to know python coding before starting this course, but you can be sure that there’s a big difference between someone who just learned hello world and for loops versus someone working at a big financial institution. Now, even in those big financial institutions, there could be big gaps in performance, even within your own team, there can be big gaps in performance. I’ve seen guys who’ve been working for 15 years do worse than guys who’ve been working less than one year. So even if I say you need to know python coding, there can be a lot of variation in skill among those who consider themselves good at it. A similar situation happens with deep learning. I can say you need to know how neural networks work, but there’s a big difference between someone who can code them from scratch and has experience using them versus someone who just watched them online animations for a few hours. If you had to bet on who knows neural networks better. Who would you pick? So it’s a really answer this question, we have to turn to the course itself, if I say you need to know python coding and you claim that you do, but you don’t understand some python code in this course, then simply put, your python coding skills are not yet up to par. So you have to improve your python coding skills by asking questions on the Q&A and taking the appropriate steps to cover your gaps. Quite often I get people who are just too confident in their own abilities when unfortunately the reality is there is a huge discrepancy between what they think they know and what they actually know. What’s funny is these people sometimes say this course requires PhD math. I assure you there’s absolutely no math in any of my courses that requires a Ph.D. or that only a Ph.D. can understand. My courses contain only undergraduate math at most. Sometimes people even claim to have a Ph.D. and then claim that it’s still too hard, which is funny because that essentially invalidates your entire Ph.D. The entire point of a Ph.D. is to turn you into an independent researcher. And so if you’re supposed to be at the level of doing independent research, but you can’t understand some undergraduate math, well, that’s not very good. Essentially, what it boils down to is people are really trying to say everything I don’t understand is not important. Everything I do understand is important. And therefore, everything you say that I don’t understand, I don’t need to know. You can see why this thinking is dangerous. If you truly believe that everything you do not know is not important, then you’ll never learn anything new. Now, this brings us to our third question, is this course fast paced or slow paced? And I think you can see where I’m going with this. This is yet another example of ambiguity. Fast pace to whom, whether it’s fast paced or not, depends on which person is taking the course. So if someone is well prepared, then things will be just right. If someone already knows most of these topics, then it will probably be too slow. If someone chose not to prepare for the course or they have an inadequate understanding of the prerequisites, then it’s going to be too fast. So if I expected you to know something and you didn’t, then you have to look it up, which is going to make the course feel like it’s going too fast. So here is the real test, if I say something depends on calculus of probability and you claim to know calculus and probability, but you don’t understand the thing that depends on calculus and probability, you have to ask yourself, honestly, do I really understand calculus and probability or did the instructor invent his own kind of calculus and probability that I don’t yet understand? Well, of course, that’s not very likely. It’s most likely that you thought you understood calculus and probability, but you really don’t. Now on to our initial question, what kind, of course, is this? Is it academic? Is it for professionals? Is a practical sort of understand this. You have to understand where machine learning is usually taught in how it is usually taught. Firstly, machine learning is usually taught in upper year computer science and engineering programs in university and college. So these are students who already took calculus one to three linear algebra, differential equations, probability and statistics, discrete math programming and possibly more all in their first two or three years of college. So by the time they get to third and fourth year, they know all this stuff and they’re ready to apply it. So a typical academic course might cover a whole library of machine learning algorithms, so you might learn K nearest neighbor, logistic regression K means clustering PCA and so on, all in the same course. Typically, you’d cover one algorithm per lecture, which is about two hours overall, you might have something like 12 or maybe 20 of those lectures in a term. And by cover, I mean mostly via math, geometry and derivations. These courses don’t typically have you at your computer doing programming work. If you do end up doing programming, it’s usually part of a lab assignment, which is actually just a very small part of the course compared to the rest. So it’s mostly theoretical dealing with equations, solving equations, reasoning about equations. Those equations typically involve probabilities and information theory. So you have two hours to go through the theory for one algorithm. And then if there’s coding involved, which there usually isn’t, you do it on your own time. The lectures themselves do not involve any coding. But there’s an even bigger problem with these academic courses and what is that? What happens is that the grades turn out to be very low. Out of 100 percent, the average might be around 25 percent or up to 40 percent. Now you might think, wow, so everyone fails. And of course, that’s not the case since everyone gets shifted based on the average. So what is the real problem here? A lot of people ask me, well, why don’t you cover every machine learning algorithm in one big mass, of course, and just call it machine learning? Well, the answer is that if you tried to do too much, you only end up understanding 25 percent of the material, which I think is clearly demonstrated here. And as they say, the proof is in the pudding. If you look at these courses where students are trying to digest 10 different algorithms at the same time, what is the result? The result is they don’t understand any of them. If you try to understand too many things, you end up understanding nothing. I take the perspective that if you’re going to take a course, you should aim to understand 100 percent of the material. Each course is broken up into appropriate levels based on how difficult they are and by topic. So, for example, you can typically take Calculus One, which focuses on differential calculus, Calculus two, which focuses on integral calculus and calculus three, which focuses on multivariable calculus. You would never learn all of these and just one huge course called math, because that’s just too much stuff to understand all at once. I want you to understand 100 percent of the material, even 50 percent or 75 percent, is not good enough, in my opinion. If you only understand 75 percent of the material and the next course depends on the 25 percent that you didn’t understand, well, then you’re not going to make it to the next level. Now, let’s look at the opposite end of the spectrum, what most people might consider to be a practical course. I don’t consider these to be practical courses, and I’ll explain why later, but I use the word practical here in quotes, since that’s what some of the more beginner students will refer to them as. So how does a practical course typically proceed? Well, first, you were never really taught how the algorithms work other than at a very high level. You might learn how they work by way of analogies and metaphors. So, for example, suppose you’re learning about gradient descent. Well, the instructor might relate this to a ball rolling down a hill. Now, some beginner students, they might conclude that I understand a ball rolling down a hill. Therefore, I understand gradient descent. But they neglect to understand how to take derivatives. They neglect to understand how to choose a good learning rate. And most importantly, they neglect how to implement gradient descent in code. So it’s totally wrong to assume that just because you can visualize a ball rolling down a hill, that you understand gradient descent, why? Well, the problem is everybody can imagine a ball rolling down a hill, your grandparents, you can’t even turn on a computer, can understand a ball rolling down a hill. So don’t equate understanding an analogy to understanding the real topic. So what’s the next thing these practical courses do, the next thing they usually teach you is how to plug into an API or in other words, make use of a library, someone else wrote. This is also not ideal. Typically, this involves just two or three lines of code. You have to wonder if machine learning just boils down to two or three lines of code. Why is it such a big deal these days? Why did it take decades for people to realize its usefulness? And of course, the answer is that there’s much more to machine learning than just these two or three lines of code. When you use a deep learning library like Carras, you’re not doing deep learning. You were using a deep learning interface. You’re simply programming. It’s more accurate to say I can program with deep learning libraries rather than saying I actually know deep learning. But some of you might say, well, it’s all I really want to do. All I need is an API and for the API to give me an answer, I don’t care about how it works. But then here is where you run into trouble. Recall that in addition to all data is the same. We know that all machine learning interfaces are the same. So the question becomes, which API do you choose? If they’re all the same, how can you make an informed choice about which one to use? Well, the reality is, if you don’t know how these things work, then you can’t answer this question. This is why some of these so-called practical courses can cover 20 algorithms in the same course. It’s because they never actually talk about how the algorithms work since the API is the same every time. It’s very easy to just do the same thing 20 times and then not bother with the details. So this gives people a lot of confidence that they were able to do the same thing 20 times. But it’s dangerous thinking because you don’t realize you really only learned one thing. You just repeated it 20 times. So how about when it comes to the professional world? Sometimes people tend to believe that in your profession, you’ll be leaning more toward the practical end of the spectrum where you don’t really need to know how a machine learning model works. You just have to know its API. But what it really comes down to is what your job actually is. If you’re a regular everyday programmer and you just want to see how a machine learning model might perform on your data, then you’re going to use a pre-built library. Most likely that’s just the two or three lines of code I referenced earlier. However, that does not mean you don’t still have to conform to best practices, make sure you’re not mixing your test set with your transfer. Make sure you’re doing proper validation and so on. And when it comes to tuning your model, then you have to be able to dig down into the details and see which hyper parameters are possible to change and which are likely to have the most impact. So only knowing how to use an API is only going to get you past the first stage. You can plug in your data and at least your code is the right syntax and doesn’t crash. But of course, your code not crashing doesn’t necessarily mean it’s right. Back on the other end of the spectrum, you have programmers and scientists whose sole job it is to do machine learning. These guys are machine learning experts. They know the math inside and out and have a full computer science education. And so to even be able to communicate your ideas effectively, you have to be comfortable with a theory and not just how to plug into an API with two lines of code. These scientists are building custom models, really digging down into the math to not only use but invent new things and to be able to invent new things. You really need to know the math very well. And so if you’re on a team like this, you can’t just know how to write two lines of code. Your skill set has to be far beyond that. So your theoretical background has to be very solid if you want to be able to communicate effectively with other scientists. Now, a lot of people say, well, I don’t ever want to be a scientist, so I don’t really care how these algorithms work. Unfortunately, that’s a very bad attitude to have in the workplace. Suppose, for example, I am running a business now, an employee of mine, he wants to show off and say he’s learning about machine learning. He says, oh, I’m taking this machine learning course online, therefore I will become a machine learning master. But his approach to machine learning is to learn as little as possible. He says he doesn’t care about the math. He only wants the practical stuff. He only wants to know the bare minimum. He tells me he doesn’t care how it works. He’s just going to plug into some API because that’s what his buddy told him he could do. As a business owner, I really don’t want this guy working for me, I run a business that sells something and the life of the company itself and the livelihoods of the employees all depend on our products being the best they can possibly be. So if someone comes to me and says, I don’t care about being the best, I just care about doing the bare minimum, my response is I don’t want that guy on my team. In fact, he should be fired. So be careful if this is your approach to machine learning. No business owner wants to hear that your approach to the business is to do the bare minimum and to put off some math because you think it’s too hard. That’s a scary prospect for a business owner. Now, if we go back to our original question, maybe you’ve realized we haven’t answered it yet. What kind, of course, is this where you can think of it as a marriage between these approaches? We want to cover all the math and derivations because that’s very important. If you want to be able to communicate effectively with other scientists, to be able to debug your code effectively and to be able to invent new approaches. This is also what we mean by machine learning. We mean really knowing machine learning and not just programming with an API. So the theoretical background is very important. At the same time, we’re not going to introduce 20 different algorithms in the same course because unlike a typical academic course where you can pass just by understanding 25 percent of the material in this course, it’s structured in a way that if you have any missing gaps, it’s going to be impossible for you to move on to the next stage. But academic courses don’t contain much coding, if at all. So in this course, every algorithm we learn about will be implemented in code. My motto is, if you can’t implement it, then you don’t understand it. You can pretend to understand it. And when you talk about it, you might sound like you understand it. But implementation is the true test. It’s like a math exam. Rather than writing an essay, when you write an essay, you might be able to fool the teacher into thinking you understand something. But in a math exam, either your answers are right or your answers are wrong. In this context, either your code works or doesn’t, it’s very clear cut. When it comes to practicality, we cover that, too, we demonstrate our algorithms both on real world data like text and images, and we demonstrate our algorithms on two dimensional data sets so that you can visualize what an algorithm is doing. Both of these are important. Text and images are quite possibly the most practical data. There is some billion dollar companies exist and thrive because of those capabilities. Duty data sets are critical as well, because as humans, that’s the only thing we can see. But at the same time, we don’t try to pull the wool over your eyes and pretend like you need 20 different data sets to practice on. Instead, we learn something much more valuable how to use an algorithm on an infinite number of data sets, because we are able to abstract the idea that all data is the same. Remember that all the computer sees is a list of numbers. It doesn’t know that those numbers represent height measurements or an image pixel. And of course, by experiencing both ends of the spectrum, both the theory and the implementation. And then finally plugging data into your model, that’s going to make you very well-rounded professionally. To conclude this lecture, I want to pose a final question, how does all this knowledge help you? Why does understanding the approach of this course make you better at machine learning in general? Well, my hope is that this gives you a very high level bird’s eye view of the landscape. You can see the different approaches of people from different backgrounds. Academic people typically don’t see what other non-academic people are doing. And those people who hate math typically don’t see what people who love math are doing. So with this bird’s eye view, you can see everybody if you’re learning machine learning, it helps you understand who am I competing with when I apply for a job and what kinds of jobs are out there and what would be adequate preparation. If you’re hiring for a machine learning team, it helps you understand the different backgrounds of individuals. You have to compare to one another to understand their strengths and weaknesses and what they bring to the table.

3. Machine Learning and AI Prerequisite Roadmap (pt 1)

Everyone, and welcome back to this class. In this lecture, I’m going to answer a question that I get pretty much multiple times per day, and that is what order should I take your courses in? This is a great question and it’s something that even I have to think about sometimes. What is the best way to learn all this stuff? So I think that what you’ll find is this ordering makes a lot of sense in that we’re always building on skills we learned before. So it’s just like learning calculus. First you learn calculus one, then you learn calculus two, which is kind of the opposite of calculus one. Then you learn Calculus three, which expands on the ideas from both Calculus one and two. And then from there you can go on to even more advanced topics like probability. So this is a process based on skill building. Now, it’s important to keep in mind that this lecture is not just about my courses in how to navigate them, although that is the question I’m answering, since that’s a question I get multiple times per day. But this is also a lecture on how different machine learning topics are related and what depends on what else. So even if you end up not taking any of my courses, it doesn’t change the fact that these topics are still related to each other in these ways. So in some sense, this lecture isn’t really about my courses at all. It’s a general lecture about how different types of machine learning models are related and in what order you might want to learn them from a skill building perspective. Now, I want to show you this stuff in a visual way. I’ve actually set up a webpage where you can go to and look at the chart I made for yourself. So what I want you to do is go to this website, Deep Learning Courses, Dotcom. Now, I want you to click on the catalog link. And from there, you want to click on the Click Here link. And so this brings us to the court’s order webpage. So at the top, you have sort of this linear chart, right, it’s just one course after the other, after the other. But this is not really what I want to look at in this lecture, this linear chart gives you a basic overview, but it’s not ideal. The reason is the relationships between courses are way more complex than this. Sometimes one course gives you the skills you need for two other courses. Sometimes there will be a course that so complex that it depends on multiple different courses. So this is the perfect opportunity to use a graph. So if you scroll up at the top, you can use this link to just jump down to the bottom and we have this graph where you can see the dependencies between each course. So what I’m going to do is I’m going to go through each link in this graph and explain to you why it exists and why it’s important. So let’s look at the actual course first, since that has no links. So it may show up in the middle so you can just drag it out to the side. That’s the coolest thing about this graph, because you can just pick stuff up and drag it around. It’s very interactive. So Escudo has no links. This isn’t a prerequisite to any other course, nor is any other course a prerequisite to it. It is standalone. You can take it at any time. Next, you’ll notice that there are courses that have no links going into them, so, for example, the no, because this is because none of my other courses are a prerequisite to this course. This, of course, was just designed to give you a basic understanding of the syntax and tools we use in data science and machine learning. The goal is actually to not do any machine learning in this course. Believe me, it was actually quite difficult to accomplish that because it’s so easy to use something in machine learning as an example. It was very hard to avoid talking about it. You’ll also notice that there are some courses with no outgoing edges. You can think of these as the most advanced courses. They are at the top of the ladder. So, for example, this one right here, of course, that could change in the future as I create more courses, in which case I will update this lecture. So let’s start with what’s coming out of non-pay. You’ll see that there are two links, one goes to linear regression and one goes to Bazhaev machine learning, a B testing. And by the way, you can zoom into this graph. So that’s what I’m going to do right now. So here’s linear regression, here’s a Bayesian resealing. So while Nampak goes to both of these courses, I actually strongly recommend linear regression as the next step. Bayesian machine learning doesn’t depend on any other courses per say, but it’s still more advanced. It requires a more advanced understanding of probability and of real world engineering problems, like optimizing the click through rate of a link on your website. So linear regression would be my number one recommendation for your foray into machine learning. Now, after linear regression, we have logistic regression. So why do we go from linear regression to a logistic regression? Well, the basic idea is both of these are linear models. In other words, the model is of the form Y equals M, X plus B, which is a line. The difference is that linear regression does regression while logistic regression does classification. We want to learn classification because that’s one of the central tasks of a neural network, which is what comes after. Now, you might notice that linear regression also has two outgoing links, so there’s one, two logistic regression and one to reinforcement learning. This is because my reinforcement learning class makes use of linear regression. So naturally, if you don’t understand linear regression, you will understand that part. But again, this is a much weaker link and going from linear regression to logistic regression. This is because reinforcement learning is actually conceptually more difficult. It doesn’t depend on the content of any other courses directly other than what we’ve shown, but the ideas behind it are more advanced. You really want to have some experience with both supervised and unsupervised learning before jumping into reinforcement learning. It helps to build the proper perspective on the subject. We will get back to reinforcement learning later. But for now, let’s go along the deep learning path. So on the deep learning path, we last discussed logistic regression. This is also known as a neuron. This is very important because in order to build a neural network, we must first know how to build a neuron. A neural network is just a bunch of neurons stuck together. So this leads us to deep learning in Python, part one, which is the bright green logo. This course teaches you how neural networks work and basic fundamental operations like back propagation. Once you know the basics of deep learning, you can move on to deep learning Part two or modern deep learning in Python. This is the same image, but in a darker color, to signify that it’s a continuation of the previous course, as the title suggests, this course is all about modern developments in deep learning. So we go over ways. We’ve improved basic back propagation like momentum and adaptive learning rate techniques like arms, prop and atom. We talk about modern regularisation techniques like drop out and batch normalization. And most importantly, we look at modern deep learning libraries like Viento intenser flow. We also take a look at Karez Pythonic and Max there and see A.K.. You’ll see that once you know the fundamentals, things don’t change that much from one library to the next, it’s all the same concept. Now, learning about these modern library sets the stage for the next level, which is applying deep learning to certain special data formats, the two fundamental types of data are text and images. These are special because they have unique structural properties and we can make use of our knowledge of those properties in order to build better neural networks. For example, we know that text is made up of sentences and sentences or sequences of words. So sequential modeling is important. We know that images are two dimensional objects, but importantly, each pixel is very likely to be the same as nearby pixels. For example, if you pick a random pixel on an image of a red car. If that pixel is red, then most likely the pixels around it are also going to be red. There is also no question about the practical applicability of text and images. The Internet is made up of text. So by doing machine learning on text, you’re learning techniques that essentially allow you to write models of the entire world’s knowledge. Images are also everywhere. Snapchat, Instagram, Facebook, self-driving cars, home security systems. Knowing how to deal with images is super important, and billion dollar companies exist because they are good at it. So that brings us to the next course. In the deep learning series. Convolutional Neural Networks or Deep Learning in Python Part three, this is all about how to incorporate convolution into neural networks and what convolution is in the first place and how that helps us make use of the structure of images. Let’s scroll up a bit. So after convolutional neural networks, we take a detour into unsupervised, deep learning, unsupervised deep learning isn’t very popular with beginners, but it should be. If you look at the experts of deep learning, like Yan Lican, Yahshua, Bengoa and Geoffrey Hinton, they all talk about how important unsupervised deep learning is. This course introduces us to several new ideas. First, the idea of latent variables or latent causes. These are factors that you don’t directly observe in your data, but that you can learn about by studying the structure of your data. Second, modern techniques for data visualization like Tesna. This will be very important since, as I discussed earlier, along with images, text is a really important data format. In order to visualize text in later courses, we will be making use of Tesna. So it’s good to know what it is and why it’s useful. Third, unsupervised, free training. Unsupervised retraining is the foundation of modern ideas like transfer learning, and we make use of it both in NLP and with convolutional neural networks. Fourth, the vanishing ingredient problem, we demonstrate the vanishing ingredient problem directly, so you understand where it comes from, vanishing ingredients are especially important in the context of recurrent neural networks. And it just so happens that recurrent neural networks is the next course in the deep learning series Deep Learning in Python Part five. So recall, there are two fundamentally interesting data types are text and images. We’ve looked at images, and in this course we start to look at text. Why might that be? Well, recurrent neural networks specialize in modeling sequences, and text is just the sequence of words. So it’s the best type of data to use to demonstrate the principles of Arnon’s.

4. Machine Learning and AI Prerequisite Roadmap (pt 2)

Once we’ve studied Arnon’s, we can go even deeper into how deep learning can be used on text. That’s the next course, natural language processing with deep learning in Python or deep learning in Python, part six. In this course, we look at a very important concept called a Werdum betting’s, these allow us to turn words which are categorical variables into vectors, which are numbers that in turn that word can read. This is why it’s very important to have studied unsupervised learning before this, because finding Werdum betting’s is actually an unsupervised task. Word embedding also allow us to make use of pre training, which was discussed in deep learning in Python part for. In this course, we also look at a very advanced model for doing sentiment analysis called a recursive neural network or a tree neural network. This is an example of a dynamic neural network because it changes its structure based on what input you give it. Most deep learning libraries are not equipped to handle dynamic neural networks. And I demonstrate what happens if you try to build one the naive way you basically end up building a separate neural network for each of your training samples and it’s going to eat up all your ram and make your computer slow down to a crawl. So you get a firsthand perspective of why working with dynamic neural networks is not easy. But luckily we are able to make use of our knowledge of recurrent neural networks. And remember, that’s the prerequisite to this course in order to convert a tree into a sequence which Anadan is capable of handling. So that’s why Arnon’s is a prerequisite to this course. Now, you notice that this course, deep in L.P, has no outgoing edges. This is because it’s the most advanced course that I have on this path. For now, I certainly plan on expanding in this direction in the future. However, this is not the last deep learning course in the series. After this, we have deep reinforcement learning, which is deep learning in Python part seven. With that said, I think now is a good time to go back and start exploring the reinforcement learning path. So you’ll notice that I have Bayesian machine learning, feeding into reinforcement learning. On the surface, these two corpses might seem unrelated, but there is a very important concept you’ll learn that applies to both called the explore exploit dilemma in Bayesian machine learning. You learn this idea in the context of trying to optimize it, click through rate or a conversion rate, or in other words, the number of times people buy things from your website versus the number of times people visited your website. Very practical concept. I think if you do anything related to e-commerce now in reinforcement learning, we look at the explore exploit dilemma again. But in the context of playing games, reinforcement learning is like a third branch of machine learning, whereas the other two are supervised and unsupervised learning. The main difference is that supervised and unsupervised learning look at static data and reinforcement learning. The idea is more like you have a robot living in the real world. It can take the experiences it had today and based on them, behave more intelligently tomorrow. So the learning paradigm and reinforcement learning is sequential. This is supposed to supervise an unsupervised learning where your data set usually resides in some file on your hard drive. So in reinforcement learning, part one, we get all the basics, as you might expect, this is a prerequisite to deep reinforcement learning, since deep reinforcement learning applies those concepts to more difficult games. But you’ll notice that this is not the only prerequisite to deep reinforcement learning. As the title suggests. This is also dependent on knowledge of deep learning, in particular convolutional neural networks. But as we know, in order to build a CNN, we have to know how to build a regular neural network, which means we have to know what a neural network is and why it’s useful and so on. So in deep reinforcement learning, these two paths converge. You combine your knowledge of both reinforcement learning and deep learning in this course. Now, the reason it depends on CNN’s and not, say, Arnon’s, which is over here, is because we’ll be learning to play visual games. So, for example, we can learn how to play a video game like Pong, a breakout, which are classic Atari games from the old days. And so those are images because they’re basically screenshots from the screen. In the future, we might end up applying Arnon’s, in which case Arnon’s will become a prerequisite to that course. So that’s the end of the reinforcement learning path for now. I’m very excited to bring you more updates in this area in the future. Let’s now jump back to you, logistic regression. Where we can see another outgoing edge to supervise machine learning, so why is this edge here? Well, you may recall that linear regression and logistic regression are both linear models that do regression and classification, respectively. These are both supervised learning tasks. And so it makes sense that now that, you know, one model for progression and one model for classification, it’s time to dig deeper into supervised learning. The thing with linear regression and logistic regression is that they aren’t really different models. They’re both just the line because they do different tasks. The techniques and interpretations are slightly different to. And there are, of course, different models that are not linear models that can do these tests, and that’s what this course is all about. In this course, we look at classic supervised machine learning techniques like Canibus Neighbor Decision Trees, the Perceptron and the baize classifier. Much like how logistic regression was the basic building block of the neural network, classic models like decision trees are the basic building block of ensemble methods. So that’s why this course is a prerequisite to ensemble machine learning, again, we use the same logo with a different color to signify that these two courses are very closely related. In ensemble machine learning. We learn how to combine multiple decision trees in different ways in order to make some very powerful classifiers. What’s really remarkable about these methods is that they are very easy to plug and play on data. So if you’re looking for a plug and play solution without having to learn a lot of theory, then deep learning is most likely not for you. But ensemble methods are a great fit. Deep learning is very dependent on hyper parameters, and if you choose incorrectly, your model will perform very poorly. Sometimes it requires immense computing power to find good hyper parameters. This is an active area of research that has not yet solved. This is why you can implement what you see in a deep learning paper. But suppose the author left out some seemingly insignificant detail so you end up having to make an assumption and then your results end up totally different. So deep learning is fragile, but luckily ensemble methods are not. We focus on two very famous ensemble methods, the random forests and the boost. So that’s everything on the supervised machine learning track. For now. Next, we see an edge going from supervised machine learning to unsupervised machine learning, in particular cluster analysis. The reason we study supervised learning before unsupervised learning is because unsupervised learning is a little more abstract. It takes more effort on the student’s part to realize why it’s practical and what it can be used for. Cluster analysis shows us how to model data that does not come with targets, as you might guess. We do this in the form of clustering. The idea behind clustering is very simple. We want to know how many naturally occurring groups of data there are and what are the relationships between the data in these clusters. So, for example, if you were clustering books, you might find a book about Steve Jobs and a book about Elon Musk in the same cluster. This cluster is probably about tech companies in Silicon Valley. But you don’t need a label in your data to tell you that you can discover it yourself by looking at how the data naturally groups together. Now, as I mentioned earlier, I think once you learn about both supervised and unsupervised learning, you’ll be ready to jump into reinforcement learning. I haven’t made cluster analysis a prerequisite to reinforcement learning, since none of the material depends on this course. But it’s good to know about these techniques so that you have a more mature and experienced view on machine learning. What is sort of a sequel to Cluster Analysis is hidden Markov models. The reason might not be clear at first, so let me give you two reasons. Number one, they are both unsupervised machine learning models, just that ACMS is harder, so it’s natural to learn about clustering first. Clustering is also about static data, whereas ECMS are about sequences. So it’s similar to the process we did in deep learning. We looked at static data like images, then sequential data like text. Reason number two, in cluster analysis, we learn about a technique called the Gaussian mixture model, which we make use of in the same course. One key point is they both learn by using the expectation maximization algorithm. So it’s good to first see the algorithm on a simple model. And then when you see them again on a more complicated model like the human, it won’t be as intimidating. One key concept you learn in acronyms is the mark of assumption that just means the current state depends only on the previous state, but not any states before it. This is a simplifying assumption that usually makes the math easier to work with. You will also notice that we encountered the mark of assumption and reinforcement learning, however, it’s not too hard to learn it from scratch. And so for that reason, I do not consider ECMS to be a prerequisite to reinforcement learning. The mark of assumption is really the only thing they have in common. There is also a slight connection between cluster analysis and unsupervised learning, so I’m not going to draw the link right now, but I sometimes consider this to be unsupervised machining part one and this to be unsupervised machine learning, part two. We also see that HMS feeds into the answer, of course, which is about deep learning. So why might that be? This is, of course, because both these models are models that can learn about sequences, in particular in both these courses with model text as sequences. But whereas the HMS makes use of the mark of assumption, he Arnon does not and hence the Arnon is a more powerful model. And so this just goes along with the main theme that we always go from simple basic models to more complex models. This is also something you should do in your work as well. If you start with a simple model, you often find that it is faster and more robust. Complex models sometimes break down, but they are also more difficult to implement and might not even be fast enough. Of course, that’s just a generalization. So you always want to analyze engineering tradeoffs individually for every problem you have. Now, there is one last link in this part of the graph here that I want to explain, and that’s the first A.P. course you can see there. It depends on supervised machine learning and feeds into deep enthalpy. The main purpose of this basic A.P. course is to apply basic machine learning models to text. So that’s why I supervised machine learning comes before it is because that this course was all about basic machine learning models. The important skill for NLP was not the implementation of those models, but rather a bigger picture perspective on how machine learning is used. What is the interface between the data in the model? What does the model do? What is its input and output? How is the output interpreted? And so we take those principles and we apply them to text. In this way we can see that text can be created in such a way that you don’t have to think about it any differently than any other data. This reinforces the principle that all data is the same, the machine learning model doesn’t care what your data is, all it sees is a table of numbers. It doesn’t care if it’s text or images or radar signals from space. The model just does what it was designed to do on the numbers that you give it. So this course gives you a high level systems perspective on working with machine learning models in text. This easy A.P. course also feeds into deep entropy, which is of course not so easy because it depends on a lot of background in deep learning. One of the main questions I get in the NLB course is how do I improve the results of these basic models? And a lot of the time the answer to that is, well, you have to use a more complex model. But of course, that necessitates learning how that complex model works. And deep OP is an example of that because we learn a state of the art method for sentiment analysis, whereas in SCMP we used on the linear model. So it’s important to understand that while, yes, it’s possible to improve the predictive ability on simple basic models, as you can see, it’s not always an easy path to get there. So you have to make sure you’re prepared. Case in point, just look at all the time spent just to get to deep. It’s not an easy task. Let’s now scroll over to Gan’s and recreational auto encoders, just like how deep reinforcement learning is not related to deep in OP, this isn’t really related to deep reinforcement learning either. This is deep learning Part eight by order of creation only. It is the spiritual sequel to Unsupervised Deep Learning, which was Deep Learning in Python Part four. OK, so just to reiterate, this is part six, part seven and part eight by order only, but they are not related to each other conceptually, although it’s always nice to know all these things because more context makes future things easier to learn. So the reason this is linked to this is because gangs and recreational auto encoders are also unsupervised deep learning models, but whereas unsupervised deep learning was all about how to improve supervised learning gains and variation, auto encoders don’t have any direct benefit to supervised learning at all, although we do make use of supervised learning within the course. In this course, the focus is on generating images. We’ve seen that Gan’s can create photo realistic images based on a dual neural network system. That’s pretty cool because before Gan’s we didn’t have any kind of machine learning model that could generate real looking images. Nowadays, Gan’s are able to generate high resolution, high quality images of people that you can’t even tell are not real people. It certainly makes the idea of the Matrix seem very possible. All right, so I hope you found this lecture helpful. We saw that these courses are related to each other in some pretty complicated ways. Learning machine learning is not exactly linear. Sometimes you have to take one course before you can take the next. Sometimes one course might just be related to another course by some key concept, but maybe in one context, it’s a lot easier to understand. So remember to keep in mind that these arrows did not all indicate strong prerequisites, or rather there’s just a relationship between the two courses. I hope that this lecture explained any nuances between what is a prerequisite and what is not. And I hope I did a good job of answering. Which order should you take these courses in and why?

15. Appendix FAQ Finale

1. What is the Appendix

Everyone, and welcome back to this class. In this section, I’m going to go over some common questions that students have and some general tips that have helped students in the past. Please keep in mind that this is not a core part of the course. These are just common general questions that students have had about machine learning or about my courses in general. I realize that it’s not useful for everyone, but please keep in mind that it’s useful for someone. There are a few people out there that tell me they think all the lectures should only answer their questions, but not anyone else’s questions. So just keep in mind that many of your fellow students have asked me these questions in the past. And in order to answer those questions in a scalable way, I’ve created the section. Please try not to hold it against me, as this may not directly help you, but it may help your fellow students. Please also keep in mind that the appendix section in no way takes away from the core content, all the material provided for the topic matter is exactly what I wanted to provide. These appendix lectures are simply extra stuff. So these appendix lectures do not displace any technical content. If the appendix section did not exist, the technical part of this course would be exactly the same as it is now. So just remember this. The appendix is in addition to the technical content. It does not displace any technical content. You get all the technical content that I intended. Also, keep in mind that you’re always able to select the lecture you want using the user interface, that’s this thing. If you don’t know how to use the video player to select a lecture, please ask me how on the Q&A or contact your support so you’re encouraged to select the lectures you want to watch manually and skip lectures you don’t want to watch. Also, keep in mind, however, that a common mistake is in thinking that this material is not going to help you when in reality it will. For example, I come across students all the time who have no idea how to get started implementing machine learning algorithms and code or why we are doing so in the first place. Then I come to find that they have never watched how to code by yourself. So I think it goes without saying that if you don’t know how to code by yourself, you should probably watch the lecture, how to code by yourself. Same thing goes for the other lectures in this section. All right, let’s get to the lectures.

2. BONUS Lecture

Everyone, and welcome back to this class. In this lecture, I’m going to answer one of the most common questions I get. Where can I get discount coupons and free deep learning material? Let’s start with coupons. I have several ways for you to keep up to date with me. The absolute number one best way for you to keep up to date with newly released discount coupons is to subscribe to my newsletter. There are several ways you can do this. First, you can visit my website, Lazy Program, or dot me at the top of the page. There is a box where you can enter your email and sign up for the newsletter. Another Web site I own and operate is Deep Learning Courses Dotcom, this website largely contains the same courses as you see on this platform, but it also contains extra VIP material. More on that later. So if you scroll to the bottom of this website, you’ll find a box to enter your email, which will sign you up for the same newsletter as you would on lazy program admin. So you only have to do one of these. Now, let’s do a small digression, because this is another common question I get. What’s this VIP material all about and how can I get it? So here’s how the VIP thing works. Usually when I release a course, I’ll release it with temporary VIP material, which is exclusive for those early birds who signed up for the course during my announcement. This is a nice little reward for those of you who stay up to date with my announcements and of course, actually read them. It’s important to note that VIP material can come out at any time. For example, I could make major updates to, of course, three years after starting it and do another VIP release. The purpose of deep learning courses dotcom is to have a permanent home for these VIP materials. So even though it could be temporary on the platform you signed up on, if you signed up for the VIP version of the course, then you’ll get access to the VIP materials on deep learning courses, dotcom permanently upon request. Here are some examples of materials you might find in the VIP sections in my Tenzer flow to course, there are three extra hours of material on deep dream and objects localization. Usually I don’t release the VIP content in video format, but this was an exception. Another example in my cutting edge, Ecorse was an extra written section on the top three algorithm. This course covered three algorithm’s in total, so the extra section gives you one more. Or in other words, 33 percent more material. Another example in my Advanced and Arnon’s course is a section on speech recognition using deep learning. In addition, there is an all new section of the course on a stock predictions or memory networks, depending on which version of the course you are taking. The reason for this is I might release slightly different versions of each course on different platforms because of how the rules on all these platforms work. I must differentiate the courses, however, since I own a deep learning courses dotcom. This is the only platform that contains the most complete version of the course, which includes all the sections. Please note that this is rare, so depending on what course you are taking, it might not affect you. All right, so let’s get back to discount coupons and free material, other places where I announce discount coupons, our Facebook, Twitter and YouTube, you might want to pause this video so you can go to these URLs and follow me or subscribe to me on these sites if they are websites that you use regularly. So for Facebook, that’s Facebook, dotcom, lazy programmer, Emmy for Twitter, that’s Twitter, dotcom lazy underscore a scientist for YouTube. That’s YouTube dotcom. See lazy programmer ex. Occasionally, I still release completely free material. This is nice if I want to just talk about a singular topic without having to make an entire course for it. For example, I just released a video on stock market predictions and why most other blogs and courses approach this problem completely wrong. That’s another benefit of signing up for these things. I get to expose fake data scientists who are really marketers, whereas I wouldn’t ever make an entire course about that. Sometimes this can be in written form and sometimes it can be in video form, if it’s in written form, it will either be on lazy program taught me or deep learning courses. Dotcom, if it’s a video, it will be on YouTube. So make sure you subscribe to me on YouTube. If I release a video, I may also make a post about it on lazy programming that me and I may also announce it using all the methods I previously discussed. So that’s the newsletter, Facebook, Twitter and obviously YouTube itself. Now, I realize that’s a lot of stuff and you probably don’t use all these platforms, I certainly don’t, at least not regularly. So if you want to do the bare minimum, here’s what you should do first. Sign up for my newsletter. Remember, you can do that either on lazy program at me or deep learning courses. Dotcom. Second, subscribe to my YouTube channel at YouTube. Dotcom slash see slash lazy programmer ex. Thanks for listening and I’ll see you in the next lecture.

2. Financial Basics

1. Financial Basics Section Introduction

In this lecture, I’m going to introduce you to this section of the course, tell you what it’s all about, and also outline what you will learn. This section is all about financial basics. Now, a lot of you join this course because you want to learn about time series analysis and machine learning with financial data. But the first step before you do any of that is to answer the question, what is financial data in the first place? Usually most laypeople have a general understanding of this. They know about stock prices. They know about general ideas like buy low and sell high. They know that what you would ideally like to do is to purchase some shares of a stock today and at some later date when the stock price has risen to sell that stock for more than they bought it for, thus making a profit. Of course, there exists very many stocks in the world and you don’t want to purchase them all. Some will succeed and some will fail. Perhaps you have some knowledge about an industry or about a company, and you believe that the stock price in that industry or that company will go up and therefore you would like to purchase those stocks in hopes of making a profit. Of course, whether or not a stock price will go up is not guaranteed. You are always making a gamble. Thus, the language we use to talk about stock prices is probability. In fact, a lot of the theory behind probability was founded on gambling. In this section, we’re going to start from the very basics where do you get financial data in typical machine learning, we are accustomed to having standardized data sets like amnesty and image then. But this doesn’t make sense for financial data. There’s no such thing as a standard data set. If you want to predict the return of a stock over the next month, then the stock price from the 1950s is not going to be very helpful. So in the real world, often what you are looking for is the most recent data. For the purpose of this course, we will be working with fixed data sets. But you want to be aware of the fact that in practice your data isn’t going to be some static CSV file. Next, we will talk about what financial data looks like in this course, we will be focused on what is called technical analysis, which uses historical stock prices and volume information. This is opposed to fundamental analysis, which uses information that is more like when an accountant would look at such as a business’s financial statements. Technical analysis is what most of us think of when we imagine doing machine learning on financial data. We have a set of time series which represent historical stock prices, and we would like to use machine learning models to forecast future stock prices or something like that. Next, we’ll look at a quantity called the stock return, roughly speaking, the return is how much the stock goes up or down in a given period. This is a very important quantity. As you’ll see throughout the course, most of the modeling and the calculations we do will be focused around the return rather than the price. So what is this modeling that we will do, one important task in finance is understanding the distribution of returns. If you understand what probability distribution your returns come from, then you can calculate things like their expected value and their volatility. Most often people assume returns are normally distributed. But as you see, this is not necessarily the case. So what kind of modeling? What we do first, we’re going to look at a type of plot called the Cucu plot, which is a graphical method of determining whether or not our data comes from a certain distribution will use quantitative methods to do this as well, which in statistics is called hypothesis testing. Hypothesis testing is an important concept in this course, and it will appear again in future sections. We’ll also look at confidence intervals as well as statistical measurements such as Skewness, keratosis, correlation and covariance. These are all important in understanding the behavior of stock prices. Next, we’ll have our first mini introduction to machine learning models. We’ll look at a model called the Gas Mixture and show why this might be a good model for modeling stock returns. Next, we’ll look at an important feature of stock returns called volatility clustering. This shows us that the behavior of a stock return changes over time and that how it changes is related to how it has changed recently. This helps us to have some predictability in the behavior of stock returns. Finally, we will look at a simple example on how to simulate the evolution of a stock price simulation is also a very important task in finance because it allows you to model all the possible futures of a stock simulation can be used for things like calculating the expected price of a stock in the future, as well as the confidence bounce on that prediction. In addition, a simulation can be used to test any strategies you have come up with for trading stocks.

10. Adjusted Close (Code)

In this lecture, we are going to look at a new notebook that will help us analyze and verify the differences between close prices and adjust the close prices for our specific data sets. Recall that this is because our data set contains both closed and adjusted closed prices, whereas the Kaggle data set is not as usual. You can look at the title of the notebook to determine what notebook we are currently looking at. So to start, we’re going to download the CCV we created earlier called ESP 500 subduct, CSFI recall that this is the data set we downloaded ourselves from Yahoo! Finance. Next, we’re going to import pandas and load in our data frame using PDF, it reads CSFI. Since we like the dates to appear on the x axis of our charts, we’re going to say index call equals date and set past dates equals true. Next, we’ll do a dot head to ensure that our data frame is in the correct format. So the dates appear as the index. Next, we’re going to grab Google’s close prices to do this, we’re going to filter our data frame to select only the rows when name equals, Google will assign that to a variable called Googs. Next, we’re going to select the columns close and adjusted close. We’re going to call the plot function in order to plot these two columns in a line chart with a figure size of 10 ten so that we can see the plot more clearly. So what do we see? Well, it appears that the close price is pretty much the same as the adjusted closed price. Why is this if we look down at the x axis, our chart definitely includes the dates from 2014 when Google had its stock split. Therefore, this helps us to verify that in our data set, the non adjusted close price already accounts for stock splits. Because of this, both the close price and the adjusted close price are the same. As an exercise, you should also verify that the Kaggle data set contains the same prices, that is, we can use the Kaggle data set without having to worry about dealing with stock splits. Furthermore, we can also verify that Google doesn’t pay dividends if Google did pay dividends. The adjusted closed price would be different from the closed price. Next, we have the same code as above to filter each rowby stock symbol and then plot the close price and the adjusted close price, except this time we’re doing it for Apple instead of Google. So what do we see? Well, again, we see that there does not appear to be any stock split shown in the graph in 2014, Apple had a seven for one stock split. Thus, if that stock split did appear on this graph, the closed price should have gone down by a factor of seven, which is quite a significant change since we don’t we again verify that the close price in our data set already accounts for stock splits and everything is relative to the latest price. Furthermore, we can see that the close price and the adjusted close price are not exactly the same. They’re very close to being the same at the latest dates, but for the earlier dates they are not the same. This confirms the fact that the adjusted closed price does account for dividends since we know that Apple pays dividends.

11. Back to Returns (Code)

In this lecture, we are going to look at how to calculate returns and code as usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So we’re going to start by downloading our data set. This is the Kaggle data set, although it doesn’t really matter which data set you choose at this point. Next, we’re going to import pendas, numpty and matplotlib. Next, we’re going to load in the data using pedigreed XXXV setting only par states equal to true, but not setting the date to be the index column. Next, we’re going to extract a new data frame by filtering all the rows where the name is Xbox. We’ll call this data from Xbox. If we use the head command, we can see that this data appears as expected. Next, we can call the plot function on the clothes column to look at the stock price as a time series. Now, to get down to the real work, we know that in order to calculate the return, we need the current close price as well as the previous close price. Now, technically, all this information is already in the data frame, but not in the format we need. What we want to have is the previous close price in the same row as the close price to do this, we can call the shift function, we pass on the value one. To say that we want to shift the close call by one will assign this new column to be priv close. If we use the head, come in again, we see that our new column of clothes has been created. Notice how all the items in the clothes column are just the items from the clothes column shifted up by one. Since there is nothing to shift up, buy one for the very first value. This value must necessarily be an. And one common question I get is, shouldn’t we shift in the opposite direction, i.e., shouldn’t the previous close go backwards and forwards? I would urge you to think about this on your own. If this is a question you have, try to determine which direction is correct based on how you’re going to calculate the returns. In the next block of code, we calculate the return as discussed previously. That’s the close column divided by the previous column and then minus one will assign this value to a column called Return. Note the use of vectorized operations here. There’s no need to do something like a for loop through each row, calculating the return one by one. If you’re trained in programming, that’s probably your first thought for how to calculate the return, given a two dimensional array of items. But luckily, pandas make this operation very easy by allowing us to calculate all of the returns at once. If we do a DFG head again to see what our new data frame looks like, we see our return. Collum note again how the first row is. Enan This must be the case since we’ve closes in again and therefore it’s not possible to calculate the return on this date. Also, notice how small the returns are, as mentioned, financial engineers are pretty accustomed to working with very small numbers like this, and that’s why we use units such as basis points. Note that there is an alternative quicker way to calculate the return. I’m not as big a fan of this method for learning purposes, since it doesn’t make you think about what calculations are happening. The way to do this is we call the change function. We pass in the argument one to mean that we want to calculate the percent change over one timestep. Let’s assign this to a new column called A Return to If We Do a Dickhead again, we see our new column return to. Upon inspection, we see that both the return and return to columns are the same, verifying that our calculation of the return is correct. Although up until now, we’ve been plotting Time series, what we are often interested in when we look at returns is the distribution of returns. One quick visualization we can do to get a feel for the distribution of a set of numbers is the histogram in pendas. This is very easy. All we need to do is call the highest function on our data frame or series object. We pass in the bins argument to specify how fine grained we want the histogram to be. So as you can see, what we get is this typical bell shaped curve. We’ll discuss more about what this means in the coming lectures. But for now, this is good enough. Another thing we can do with our series of returns is calculate statistics such as the sample mean and the sample variance. To do that, we simply call the mean function and the stored function. The STD function actually gives us the standard deviation, which is the square root of the variance. This is a bit better in this case, since the standard deviation has the same units as the original data. As you can see, the return is very small, very close to zero, and the standard deviation is also quite small, about zero point zero one. Now, since we learned about log returns as well, let’s try doing the same process, but on the log returns, luckily numpty operations, broadcast overspenders, data structures. So all we need to do is take the return, call them at one and call and update log. There are other ways to do this as well, such as taking the log of the price and then taking the difference. But I’ll leave that to you as an exercise. So let’s do a DFG head to see what we get, you recall that in our previous lecture on returns I talked about how when X is very small, it’s approximately equal to log of one plus X, and we can see that kind of behavior here. Notice how the log returns are actually very close to the non log returns. They only differ in about the last two decimal places. Next, we’re going to plot a histogram of our log returns using the highest function, as you can see, we get pretty much the exact same distribution that we got for the non returns. You might want to try going back and forth between the two charts to verify that for yourself. Finally, we can calculate the sample mean and the sample standard deviation of the log return again using the mean function and the stored function as before, we see that the mean is very close to zero and the standard deviation is about zero point zero one.

12. QQ-Plots

In the previous lecture, we noticed that the distribution of returns took on a shape that looks like a bell curve. Whenever you see a bell curve, you should automatically think about the normal distribution, also called the Gaussian distribution. If you are not from statistics in this lecture, well, you’re going to start discussing different ways that we can check if the data that we have sampled actually comes from the normal distribution. This lecture will look at a visual method of checking this called the Cucu Plott short for Quansah Quanti apply. So what is a consequence, quants apply? Well, first, we should probably review what a quanti is. If you’re not a student of statistics, it’s probably a strange word. Basically, you can think of quintile as something like the inverse of the CDF. Remember, this is just intuition. So nothing I’m saying right now is meant to be precise. As you recall, the CDF, which stands for cumulative distribution function, tells you the probability that the random variable is less than the value that you pass in. So, for example, suppose I’m measuring the heights of students in our classroom. I have a clear function and I pass on the value six feet if I get back the value. Ninety five percent. That means 95 percent of the class is less than six feet tall. The percentile, which is a kind of quintile, is basically the opposite of that. I can input 95 percent and I get an answer and fee. For example, I can ask how tall do I have to be so that I am in the 90th percentile? That’s equivalent to asking how tall do I have to be so that 95 percent of the class is shorter than me. And the answer is, of course, six feet quintiles are a generalization of percentiles for percentiles. We have one hundred buckets so we can ask how tall do you have to be to be in the 90th percentile or the 96 percentile and so on. Quintiles can be broken up into any number of bins. So for example, if we have four bins, we call them quartiles. Anyway, this is not so important for understanding Cucu plots. I just want to give you a basic idea of what the word quintile means. So what does that, Quantock, want to apply, a Quansah Kwanzaa plot is a plot war on one axis. We plot the sample Quantrill’s and on the other axis we plot the theoretical quantize from the distribution that we think our data was sampled from. The basic idea is if the data really did come from this theoretical distribution, then the points on the Cucu plot should be nearly a straight line. So the interpretation of a Cucu plot is pretty simple. If you see a straight line, that’s generally a good thing. That means your data really did come from the distribution you thought it did. If you don’t see a straight line, that’s generally not a good thing. That means the distribution you try to match your data to is probably not a good fit, by the way. Note that another name for this kind of plot is the probability plot. So if you see a function called probability plot, it probably refers to the same thing as the Cucu plot. We often speak of the normal probability plot, which is a probability plot or a Cucu plot where the theoretical distribution is the normal. Let’s look at what would happen if your data does not fit the theoretical distribution. Here’s a common scenario in finance where your data is more fat tailed than the normal. That means your returns are more extreme than what would be predicted by the normal distribution. We know this to be true because sometimes the stock market crashes or sometimes things go up in value very quickly, such as cryptocurrency. The probability of these events under the normal distribution would be infinitesimally small. So we know that normal distributions are probably a poor approximation to real world returns. In any case, what does a Cucu plot look like when we have heavy tails? Well, we see that the points will deviate from the straight line at both ends. Generally speaking, there are only a finite set of patterns you will typically see in a Cucu plot when the data doesn’t match the theoretical distribution. This happens when the data has heavier tails than the theoretical distribution or when the data has lighter tails. You see a different pattern when the data is more or less skewed or more right skewed. Note that depending on which book you’re reading, what you see will depend on whether the theoretical distribution is on the horizontal axis or on the vertical axis. If you switch the axes, then you also have to flip the plots. The plots from this image correspond to samples being on the horizontal axis and the theoretical distribution being on the vertical axis. As an exercise, you might want to generate your own data to demonstrate that these patterns do in fact correspond to these scenarios.

13. QQ-Plots (Code)

In this lecture, we are going to look at how to generate and analyze Cucu plots in code, which will help us understand whether or not our daily stock returns are normally distributed. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Before we make any Cucu plots, I want to start by simply drawing the normal PDF over the histogram. We know that theoretically the histogram will approach the true distribution as the number of samples collected approaches infinity. So if the distribution we choose is a good fit, then it’s histogram should match up pretty closely with the true distribution. We can start by importing the Noor module from Zippi Dots. That’s next. We can create a list of X coordinates which will span from the minimum return to the maximum return, with one hundred points in between, as you recall. This can be accomplished with the Lin space function. Next, we’re going to generate the normal PDF with mean and standard deviation equal to the sample mean and sample standard deviation of our returns. We can accomplish this by calling the function normed PDF, the first argument is the X coordinates. The second argument is the mean and the third argument is the scale, which for the normal distribution is the standard deviation. Next, we can draw a plot of our PDF along with the histogram, simply by calling the plot function and the hist function separately. Note that for the highest function we need to pass in the argument, density equals true so that the histogram is normalized by default histogram. So the raw counts, which of course will be much larger than the values of the PDF. So what do we see, as you can see, this is probably not quite a good fit. It’s reasonable, but there are some areas of the plot which don’t look nice. For example, in the center, the data has a much higher frequency than predicted by the normal distribution. There also appear to be pretty significant gaps in the shoulders of the distribution. And thirdly, the returns seem to take on pretty extreme values, which should be very unlikely according to the normal distribution. Let’s now see how we can generate a Cucu plot to verify what we’ve seen, one method of doing this is to simply use the proper function from zippi that stats. As you recall, another name for the Cucu plot is the probability plot in the next block, we call the plot function. The first argument is the data. That’s the Starbucks returns, but we call drop in a first so that we only pass in actual numbers. Next, we say this equals Naum. To say that we want to compare our data with the normal distribution we say fit equals true so that the function will find the best parameters for the normal distribution that will match with our data. Finally, we pass in a plot equals party, which corresponds to matplotlib, which we imported earlier. So if we look at the probability plot, what do we see? Well, we see pretty significant divergence at the ends of the probability plot. This suggests that our data has much heavier tails than expected if it came from the normal distribution, which confirms what we were seeing earlier. The next thing I want to do in this script is to show you how to make the exact same Cucu plot by using stat’s models rather than zippi, although Zippi is great at doing a few statistics, calculation’s stats models is much more expansive. If you come from the language or you have a statistics background, you may have wondered if there is any analogous functionality in Python. And of course, the answer to that question is stat’s models. I want to introduce you to stats models now, since it will be a pretty significant staple in this course, especially when we look at TIME series models. So first, we need to import stats models. The usual way of doing this is to say import stats models, IP Azem sticking with our two letter convention for major libraries. Notice how we get this strange warning, this is not because of our code, since we haven’t actually done anything. This is because of standard models code. Unfortunately, that’s models is not as popular as other libraries, such as an umpire tends floor by torch. So you’re going to see things like this from time to time. If it bothers you, I would encourage you to look at the source code. Figure out how to fix the issue and then make a pull request on the stats models repository. All right. So anyway, back to our Cucu plot. This is as easy as simply calling smokier Occupy. As input, we pass in our data again, first calling drop in to remove any no values and line equals S.. So what does line equals? S mean? S means that the line is standardized. That is, its scaled and shifted by the standard deviation and mean of our data. There are other possibilities such as R, which means to fit a regression line if you want to learn about the different arguments you can pass into this function. I would encourage you to read the documentation. In any case, what do we see? Well, not surprisingly, we get a plot that looks pretty much exactly the same as what we had earlier. Aside from a few minor differences, such as the axis labels, of course they are both Cucu plots. So this must be the case. Now, as you know, in finance, sometimes we like to work with the log returns rather than the returns. So let’s see if anything changes when we use the log returns instead. Again, we’re going to create a list of X values for the X axis on our PDF. We now use the min of the log returns in the max of the log returns. Again, we have one hundred points. Next, we call normed PDF, again, passing in the X values, the mean of the log returns and the standard deviation of the log returns, we assign this to a variable called the Y list. Next, we plot X list and Y list alongside the histogram of the log returns. So what do we see? Well, we see that the picture pretty much looks exactly the same. How can this be? Well, recall that when the values of the returns are very nearly zero, adding one and taking the log does not change its value by a lot. In other words, X is approximately equal to log of one plus X when X is near zero. Again, we see the same pattern where the histogram is taller than the theoretical distribution and has much more extreme values than the theoretical distribution would admit. If we look at the Cucu plot, we again see the same pattern, the points diverge at the ends because it has heavier tails compared to the theoretical distribution.

14. The t-Distribution

In this lecture, we are going to continue our discussion about the distributions of returns and log returns, as we saw previously. One problem with fitting the normal distribution to returns is that the returns have much heavier tails than the normal distribution would predict. So perhaps the normal distribution is not such a good assumption. Now, as an aside, let me say that although this is evident, the normal distribution is still widely used as a model for returns in finance. So don’t be surprised when you see it even at very high levels of study. Well, one thing we can do is ask ourselves, do we know of any distributions that look like what we saw, we saw a bell curve, but with heavy tails. In fact, one well-known distribution that’s shaped like a bell and has heavier tails than the normal distribution is the T distribution. The T distribution is famous for its use in one of the most commonly applied statistical tests, the T-test. So what is the T distribution? Well, as mentioned, the T distribution is a real value distribution that goes from minus infinity to plus infinity and is shaped like a bell curve, but with heavier tails than the normal distribution. In fact, if you look at the equation for the T distribution, you can see that it has polynomial tails. That is the probability density drops off as a polynomial function of X. This is opposed to the normal distribution where the probability density drops off exponentially with the negative of X squared. If you are mathematically inclined, you should be able to see this right away. Otherwise, if you are intimidated by equations, you might want to go through this more slowly in order to get a better understanding of what I just said. Note that in its usual form, the T distribution has only a single parameter called NU, which refers to what is called the degrees of freedom. It’s kind of a strange name, so don’t worry about it too much. It’s a parameter just like how MU and Sigma are parameters of the normal distribution. In this usual form, the T distribution must always be centered at zero, which seems somewhat restrictive. In practice, we can extend the T distribution so that it has location and scale parameters. That is, we can move the center, left or right, and we can make it fatter or skinnier. Of course, it’s a little more complicated than the normal distribution since we also have this parameter new, which also controls how fat or skinny the distribution is. In fact, the formula itself isn’t very important for us since we can’t estimate the parameters of a T distribution and closed form anyway. This is unlike the normal distribution where Mbewe is just the sample mean and Sigma squared is just the sample variance. Instead, it’s more like a machine learning model where we specify that our model is a T distribution and then we call the fit function, which finds the parameters that best fit the data. You might guess that this uses maximum likelihood estimation. After calling the fit function, we get back the degrees of freedom, location and scale parameters of the T distribution, which we can use later for drawing PDF and Cucu plots and so forth. And by the way, note that if you standardize the random variable by subtracting the mean and dividing by the standard deviation, passing that into the T distribution with location zero and scale equals one that’s equivalent to calling the T distribution with the same degrees of freedom and the given location and scale parameters.

15. The t-Distribution (Code)

In this lecture, we are going to look at how to use the T distribution to model our returns in code. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Basically, we’re going to follow all the same steps as we did on our previous scripts, except that now we’re going to do them with the T distribution instead of the normal distribution. To recap, we’re going to plot a histogram alongside a plot of the PDF of a T distribution where the parameters of the T distribution are the parameters of best fit given the data. Next, we’ll look at the Cucu plot of the data against the same T distribution. Then we’ll repeat those two steps with the log returns instead of the actual returns. So first we’re going to import the T module from zippi that stats. I put a comment here to be careful because as you know, in programming we often like to use short variable names. It would be very easy to overwrite the variable T with something else, like a loop index. So make sure that when you are using the T distribution that you don’t use the letter T for anything else. Next, I want to do the same thing I did in the previous script, where I create 100 evenly spaced points between the minimum value and the maximum value to represent the x axis of our plot. Next, I’m going to call teet fit to get the parameters of the best fitting t distribution to our returns, data of assign these to a variable called parameters. If we print out programs, we can see that it’s a tuple containing three values. Now, it’s not clear what these three values represent. In fact, the documentation doesn’t explain what they are. So it’s up to us to find out. Let’s assume that they are in the order of degrees of freedom, location and scale, and then if we’re wrong, then our plot will look bad. So next, we’re going to get the PDF of the T distribution using these parameters as before, the first argument is the X values and the next few arguments are the parameters. Next, we plot the PDF and the histogram side by side using the same code as before. So clearly, this is very exciting. We see that the T distribution is a much better fit than the normal distribution. The peak of the distribution almost perfectly captures the peak of the data. Furthermore, there is no gap in the shoulders of the distribution as there was with the normal. And that’s because that weight is being distributed across the tables and the head. The next thing we want to do is the Cucu plot. So let’s try Stach models first. Now, you might assume, given the documentation, that we can simply pass in the T module for the defense argument. The reason we didn’t need to pass in anything before is because the default is the normal distribution. Now that we want to compare it to a distribution other than the normal, we need to specify which distribution explicitly. But unfortunately, we get an error. Notice, it’s not because we passed in the wrong thing. In fact, the documentation it tells us to pass in something exactly like this, as you can see, it’s actually calling functions on the module that we passed in, which means that what we passed in at least has the correct function names. So if we look closely, it looks like we’re trying to call a function at PPF, but it’s missing an argument for D.F., which is the degrees of freedom, by the way, you also want to be careful when you’re working with variables called D.F., because we often use the F four degrees of freedom in addition to often using D.F. for data frames. In any case, this makes complete sense. It’s just a very strange API. The reason it’s complaining is because all of the CPA functions for the T distribution require an argument for the degrees of freedom. We see this explicitly when we look at the documentation distributions like the normal only have a location and scale parameter. So when we pass in, the normal Cucu plot works just fine to get you more comfortable with this idea. Think of another distribution like the exponential. If we check out the documentation for the exponential, we see again that it only has parameters for the location and scale. There are no extra required parameters. Now, you might say this is because we’re trying to mix up zippi and stat’s models, maybe it will work if we use pure saipov. So let’s try the same thing again with the proper function from Zippi. Again, we pass in the value T for the argument dist note that this argument also typically accepts strings so I can pass in the string T instead of the majority. Basically, the string you pass in just has to match the name of the module. Unfortunately, we see that we get pretty much the same error, missing one required positional argument, D.F. So what can we do? Well, we can give these functions the arguments they expect. Specifically, they want to be able to call functions inside the T module, but without the D.F. argument. We can accomplish this by creating our own custom class. I’m going to call it mighty inside this class. I’m going to first declare a constructor, which takes in one argument, D.F., this is going to store D.F. as an instance variable so that it never has to be passed in when we call any subsequent function. The nice thing about Python is that the syntax is the same everywhere we can call traffic and trumpf, and it doesn’t matter that T is an object or a module. So that’s why this works, even though the original T is a module and what we are doing is creating an object, these functions are accessed using the exact same syntax. So next I declare a function called fit, which simply calls Tophet. Note that this doesn’t actually require a date parameter, but it’s called by the Cucu plot function. So it’s necessary to implement. Next is the important one, the PPF function. This one is only allowed to take in a location and scale, but internally we add the death parameter by passing in self-drive. So if we try s.m Cucu plot again, but passing in an object of type might with the degrees of freedom we found earlier, we see that this now works. And of course, as our density plot suggested, the t distribution is quite a good fit, the points are now not diverging at the ends any longer. Next, we can do the same exercise, but on the log returns again, the first thing we do is create one hundred evenly spaced points between the minimum and maximum to go along the x axis of our chart. Next, we call teed up fit to get the parameters for our T distribution. Next, we assign the parameters to variables called D.F. LOC and scale. Next, we grab the values of the PDF by calling t that PDF parsing in our data and our parameters. Next, we plot the histogram of the log returns along with the PDF, again, observing that this is a much better fit to the data. Finally, we create our Cucu plot again, using a mighty object for the best argument, as you can see, this is again an excellent fit, which is not surprising since the log returns are very close to the returns.

16. Skewness and Kurtosis

In this lecture, we are going to discuss another important characteristic of probability distributions that will help us understand stock returns. Let’s temporarily bring this back to the world of finance. In finance, we’re dealing with money. This is often people’s life savings. In other words, it’s pretty serious. We want to put great effort into avoiding disastrous outcomes in terms of investment. What you want to avoid is that stock dropping in value by a significant amount. So far, we know this is possible because we’ve seen that stock returns can take on pretty extreme values, more extreme than predicted by a typical normal distribution. Of course, there are real world examples as well. There are several stock market crashes that we can look to for evidence. Sometimes stocks simply drop in value and become worthless. Not every company is a Google or a Starbucks or an Apple. We tend to forget about those companies simply because they aren’t popular, out of sight, out of mind. However, this introduces a bias whereby we can only recall successful companies and we’ve completely forgotten or never even knew about the ones that failed. So far, we’ve looked at mean and variance as ways to characterize the distribution, these tell us about the expected value of a return and how spread out it is. In general, the mean and variance are known as the first and second moments of a distribution. We can have higher moments as well, like the third moment, the fourth moment and so on. In fact, these moments are the ones we’re interested in in this lecture. The third moment is called the Skewness and the fourth moment is called the Critz’s. Note that in general, the standardized moment of a distribution is the expected value of the standardized ran a variable to the power p. We say that it’s standardized because we take the rain a variable, subtract the mean and divide by the standard deviation before raising it to a power. So, in fact, the Skewness is more precisely the third standardized moment and the keratosis is the fourth standardized moment in. So why are the Skewness and keratosis important, although the equations look somewhat scary? They are pretty intuitive in both cases. We can think of these concepts relative to the normal distribution. First, we recognize that the normal distribution is symmetric. Skewness is a measure of the asymmetry of a distribution. We say that a distribution is left skewed when it has a tail on the left. The value of the Skewness in this case will be negative. We say that a distribution is right skewed when it has a tail on the right. The value of the Skewness in this case will be positive. The normal distribution has a skewness of zero. So why is this important when we look at stock returns? Well, clearly we want to avoid returns which have negative skewness. This is bad because it says that we can potentially get returns which are very negative. On the other hand, we like when a distribution has a heavy positive tail, that means we are more likely to see high positive returns. So what about keratosis, keratosis measures how heavy the tails of a distribution actually are, keratosis allows us to quantify the heaviness of the tails in a single number. Again, we usually talk about these concepts relative to the normal distribution. As we discussed previously, the t distribution and returns in general have heavier tails than the normal. Well, it turns out that the normal distribution has a courtesies of three. That’s kind of interesting as it doesn’t depend on any of the parameters of the distribution. The courtesies is just constant. Because of this, statisticians often speak of the excess keratosis. That is how much keratosis beyond the normal. It should make sense then that the excess courtesies is defined as the fourth standardize moment minus three. So if we have an excess keratosis that’s greater than zero or just keratosis that’s greater than three, we say that it has heavier tails relative to the normal. If the excess courtesy is less than zero, then it would have lighter tails again, bringing this back to finance. Generally speaking, a high keratosis is bad because it means that your returns take on extreme values in finance. We refer to this as risk. When we take on more risk, it’s possible to win big, but it’s also possible to lose big. Finally, I want to make a note about how we can actually calculate Skewness and courtesies in code if you want it to do so manually, we won’t since these functions are built into pandas. But I think it’s worth discussing. As usual, we can always estimate expected values with sample estimates or Montecarlo estimates. I think these formulas should be pretty intuitive. First, we estimate the mean Mbewe using the sample mean, which we call X Bar, we also estimate the standard deviation sigma by taking the square root of the sample variance. We’ll call that X. Finally, we combine these into a single expression to get the sample skewness and the sample keratosis. Of course, the difference between these two is that we either take the expression to the power of three or the expressions of the power for note that and these expressions you could divide by and minus one instead of N in certain places in order to get the unbiased estimate rather than the maximum likelihood estimate. The last thing we’re going to do in this lecture is to look at the actual code for Skewness and courtesies, we’re not going to do a separate lecture for this since they’re basically just a single function call. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. As promised, all we need to do is call single functions. These are SGU and keratosis. You can see that we’ve done this on both the returns and the log returns. So here’s the SKU for the return. He was there, keratosis for the return. Here is the SKU for the log return. And here’s the keratoses for the log return. So what do we see? First we see that the skew in both cases is negative. Therefore, we would say that Starbucks returns are slightly left out. You’re more likely to get extreme negative values. Also, in both cases, the excess keratosis is greater than zero. That is the courtesies for Starbucks returns suggests that they are more heavy tailed than the normal distribution. That means we should expect you get more extreme values in general. Now, just this, a sanity check, since it may not be immediately obvious, we should look at the Skewness anchor ptosis values for samples from the normal distribution, since we see that both of these values are very close to zero. This confirms that we are indeed looking at the excess keratosis and not the courtesies. If we were looking at the keratosis, then the sample keratosis should be around three.

17. Confidence Intervals

In this lecture, we are going to discuss confidence intervals so far, we’ve looked at various statistics on the returns, such as its mean variance, Skewness and keratosis. Whenever we estimate something, we also want to ask, how confident can we be in that estimate? Generally speaking, the more data you collect, the more confident you can be. So when you have more data, your confidence interval will become smaller. When you have less data, your confidence interval will become larger. This lecture is not meant to be a rigorous derivation of confidence intervals, but rather just a bit of mathematical intuition only. Before we dig into confidence intervals, I want to discuss why this might be of interest in the context of finance. Again, we are interested in the return and specifically the expected return. You’ve seen that one way we can estimate the expected return is to calculate the sample mean. Suppose we get a no bigger than zero. You might automatically assume this is a great stock. It’s expected return is above zero and therefore I will make money if I invest in this stock. However, keep in mind that this is only an estimate. What if we were to also include a confidence interval on that estimate? And what if we find that this confidence interval contains negative values in this case, we can’t rule out that the expected return could actually be negative and not positive. So the rest of this lecture will focus on how confidence intervals actually work and how we can compute them to start. We have to establish that our sample estimate of the expected return is random. That is to say, it’s a random variable. Obviously, it’s not as random as the result of a coin toss, but it’s random nonetheless. Recall that the sample mean is nothing but the sum of each of the samples divided by N where n is the number of samples collected. To help you see this more clearly, suppose I have only two random variables, X and Y, out of those two random variables, I create a new random variable Z, which is the sum of X and Y. Well, if X is random and Y is random, then surely you would agree with me that Z would also be random. For example, you flip to coins and you add up the result. Hence, the sum of random variables is also a random variable. In fact, any function of random variables is also ran a variable. It turns out that if you have two random variables, X and Y, which are independent and identically distributed normals with me, MU and Variance Sigma squared, we can easily calculate the distribution of their sum. In fact, if we say Z is equal to X plus Y and Z also comes from the normal distribution, this normal distribution has been at Tumut and Variance two times Sigma Square. If you’re interested in why this is the case, I would strongly encourage you to go through the calculation to find the distribution of X plus Y. Following this pattern, if we had summed up in different ID samples of X and called that Z, then the distribution of Z would be normal with mean end times view and a variance and a time sigma squared. After a bit of algebra, you can show that the distribution of Z divided by N. In other words, the sample mean is also a normal but with mean mu and variance sigma squared over N. Let’s take a moment to think about why this makes sense. The meaning of X is mu, so the mean of the sample mean is also mu. That’s pretty logical. What about the variance? This also makes sense. It says that as I collect more and more data, the variance of the sample mean decreases. The more data I collect, the skinnier the distribution around the sample mean becomes. This also makes sense. Since the distribution of the sample mean is normal, I can plot the PDF of this distribution from here, the confidence interval is obvious. As you know, the total area under the curve is one hundred percent. The 95 percent confidence interval is simply the middle area of the curve. That will cover 95 percent of that 100 percent. In other words, this leaves 2.5 percent on the left side and two point five percent on the right side. This is because 95 percent plus two point five percent plus 2.5 percent equals one hundred percent. As an exercise, you might want to try this yourself using Python. Suppose that I have a standard normal that is the normal with mean zero and variance one. I claim that the point on the left boundary of the confidence interval is minus one point nine six. And the point on the right boundary of the confidence interval is plus one point nine six. Obviously there is some rounding error, but these are convenient values which are usually used by statisticians. As you recall, these are called Quantrill’s, if I pass in zero point zero to five into the Quansah function, I get back minus one point nine six. If I pass in zero point nine seven five into the Quansah function, I get back plus one point nine six. Therefore, if I take the integral or in other words, the area under the curve of the standard normal from minus one point nine six to plus one point nine six, I will get zero point nine five or ninety five percent. All right, so how can we relate this back to our estimate of the sample mean, which has mean Mu and Sigma squared over N? Well, it takes some algebraic manipulation and probabilistic reasoning, but if you write this down on paper, you should be able to derive this on your own. First, we can start with the fact that we already know that X Bar, our estimate for MU has been mu and variance sigma squared over N. This is the same as saying that X bar minus MU is normally distributed with mean zero and variance sigma squared over N. So the scene has changed, but the variance hasn’t, of course, if you subtract me from a distribution with Mean Mbewe, then the new mean will be zero. The next step is to divide X bar minus view by it’s a standard deviation that’s sigma over the square root of N. As you recall, subtracting the mean and dividing by the standard deviation is called standardization in statistics. And we know that the result is that this now has the standard normal distribution. In other words, mean zero and variance one. Using what we just found, we can express the 95 percent confidence interval as follows. We know that the 95 percent confidence interval is the standardized bar we had before with a lower bound of minus one point nine six and an upper bound of plus one point ninety six. Now, all we need to do is rearrange this expression to isolate MU the true meaning. Once we do this, we obtain the familiar expressions for the lower and upper bound. The lower bound is X bar minus one point nine six times sigma divided by the square root of N, the upper bound is X bar plus one point nine six times sigma divided by the square root of N. One important thing to mention is that in reality, we do not actually know Sigma because of this, our standardized X bar must be expressed as follows with the estimate of the standard deviation s rather than the true sigma, which we do not know. Unfortunately, what happens when you include this estimate is that this random variable is no longer normally distributed. In fact, it’s t distributed. Luckily, we’ve already worked with the T distribution. So you are comfortable with it at this point, the normal approximation is justified due to the central limit theorem as well as a few other theorems which are outside the scope of this chorus basically as and approaches infinity. The T distribution approach is the standard normal distribution. In other words, what I’m saying is that even though our standardized X bar is t distributed and not normally distributed, we will continue to use the boundaries minus one point nine six and plus one point nine six. This is a justified approximation. Furthermore, I’ve played a little trick on you, what is the trick? Well, we started this lecture by assuming that our returns were normally distributed, but we already know that this is not exactly true. However, we can again apply the central limit theorem, since the sample mean is a constant times the sum of random variables. And as you know, the sum of random variables approaches a normal distribution anyway. This is actually a powerful approximation. It means that we can use the normal approximation for confidence intervals. The same one I just showed you for any distribution, even coin tosses which can only take on the values at zero and one. As a side note, another common way to estimate confidence intervals in finance and at times series forecasting is to use bootstrapping. The basic idea is we repeatedly draw samples from our own set of samples. It’s an odd idea, but it turns out to work. We won’t discuss this method any further for now, but be aware that there are other options if you’d like to research them on your own. All right, so that concludes our lecture on how to calculate confidence intervals, which will be applied to our expected return estimates to bring this back to the big picture. This is all part of trying to model returns. Previously, we looked at how returns might be distributed. Now we are looking at different ways of quantifying this distribution.

18. Confidence Intervals (Code)

In this lecture, we are going to look at the code that will look for the confidence interval of our stock returns. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. To start, we’re going to convert all of the Starbuck’s returns into a Nampara. We’ll call this array values. Next, we’re going to grab the mean and the standard deviation of the values array, we’ll call these M and S. Next, we’re going to calculate the lower and upper limit of the confidence interval, as you recall, this is the sample mean plus or minus one point nine, six times as divided by the square root of N. Next, we are going to plot the confidence interval on top of the histogram of the returns. First, we need to actually plot the histogram of the returns. So that’s the function, same as what we had earlier. Next, I’m going to call the function ASV line to draw vertical lines on the plot. The first vertical line will be located at the location m. I’ll label this with the string mean and make it the color red. Next I’m going to make a vertical line for the lower limit. I’ll call this low and I’ll make the color green. Next, I do the same thing for the upper limit. Now if you look at the plot, it’s kind of hard to see. That’s because the range of values is large relative to the confidence interval. So in the next flight, I’m going to plot only the vertical lines, so this code is the same as before, except without the histogram. I’m also going to plot a vertical line at the location zero and I’ll make this line blue. If we look at the plot, notice that the confidence interval from the left green line to the right green line includes the value zero. It also includes negative values. In fact, this is closely related to statistical testing, which is what we’re going to discuss next.

19. Statistical Testing

In this lecture, we are going to discuss statistical testing, which ties in nicely with two things we previously discussed earlier. We looked at how to check if our data was actually coming from a normal distribution by using a Cucu plot. This was a graphical method of checking whether or not your data comes from a certain distribution. Statistical testing offers a quantitative way of doing the same thing. In addition, we’ll see how statistical testing follows directly from our discussion of confidence intervals. Before we get into any theory, I want to discuss the API approach to statistical testing. This approach might be thought of as the lazy approach, the beginners approach, or just an approach for people who don’t like math or whose main field is not statistics. For these people, the process is quite straightforward. The basic API of a statistical test is like this. You start out with what’s called the null hypothesis itself helpful to use examples. So let’s suppose you are a drug company and your job is to test whether or not a drug is working. Typically in these experiments you have two groups of people. One group of people is called the control group and one group of people is called the treatment group. The control group receives a pill, which is called a placebo. Typically, this is just a sugar pill. This has no medicinal effect. The treatment group receives the actual drug you want to test. Your data will consist of some measurement which is related to the intended effect of the drug. For example, if it’s a drug to lower blood pressure, then your data would be a measurement of the patient’s decrease in blood pressure after taking the drug. All right. So what’s the null hypothesis? The null hypothesis would be that the drug has no effect. We typically use the symbol age not to express the null hypothesis. The alternative hypothesis would be that the drug does have an effect. We typically use the symbol H one or J to express the alternative hypothesis. After consulting with your colleague about the correct statistical test to perform, you then call that function in your code. The output is what is called a P value. The p value is a probability. We’ll discuss what it means later, but for now, this explanation is for the beginner approach. Only if the P value is less than some threshold, then we reject the null hypothesis. Typically, this threshold is set to a value like five percent or one percent. If the P value is greater than this threshold, then we do not reject the null hypothesis. In our example, a P value less than five percent would indicate that there is a significant difference between the treatment group and the control group. If the P value were very large, say, 90 percent. Typically, the way this would look in the data is that there would be a large overlap between the treatment group and the control group’s measurements. In other words, the difference between the treatment group and the control group is not significant. Note how there’s a subtle use of language here, if this were machine learning, we would usually speak in more concrete terms. For example, I classify this picture as a dog or I classify this picture as an airplane. But we don’t say things like I reject that this picture is a dog or I cannot reject that. This picture is a dog. Such statements would be quite silly. However, that’s exactly what we do in statistics. We don’t accept the alternative hypothesis. We only reject the null hypothesis. Furthermore, if we get a P value, which is very high, like 90 percent, we especially do not accept the null hypothesis. Sometimes people will say they accept the alternative, but they would never say that they accept the null. We can only fail to reject the null hypothesis. So just keep this in mind. When you’re talking with statisticians who tend to be extremely precise when discussing these kinds of things, you either reject or you fail to reject, but you do not accept. Practically speaking, at the end of the day, you do have to decide that the drug works or not. So it does seem quite silly that you’ll never make a statement like this. Drug works practically. That’s what you’re doing. But you prefer to say, I reject that this drug does not not work. Let’s now try to understand what we are doing in a graphical way, the previous example I gave you is called a two sample test. That’s because your control group is considered one sample and your treatment group is considered the other sample. You can imagine that your measurements from each group gives you two separate bell curves. Of course, there might be some overlap. For example, if you’re measuring the heights of men and women, of course, there are some women who are taller than some men. So there is some overlap between the two groups. However, the difference in height between men and women is significant. So what a statistical test is trying to do is tell you whether these two bell curves are significantly different. A naive approach might be to look at this and say, well, why can’t I just take the sample mean of each group and compare that? If the sample mean of the treatment group is larger than that of the control group, then the treatment works. Of course, that does not account for statistical flukes. Think of an extreme example where your control group only has one person and your treatment group only has one person. If you only compare sample means, then one groups mean must necessarily be larger than the others. So if the control group has been zero point zero zero one and the treatment group has been zero, that doesn’t necessarily mean that the treatment makes your condition worse than the control. In fact, this doesn’t account at all for the fact that the treatment might have no effect depending on the variance and sample size. It may simply mean that there is no detectable effect. So the two sample test is nice for intuition, but the one sample test is nicer for understanding the mathematics. The one sample test is even simpler than the two sample tests. In this case, I only have a single group of measurements and what I want to do is compare them to a single number. This is opposed to the two sample tests where I want to compare the two groups of measurements to each other. For example, suppose I want to compare the scores on a standardized test for my school compared to the national average GraphicLy. What I want to do is draw a bell curve representing the distribution of the sample mean of the test scores along with a point representing the national average. The national average will be called Muhanad and the average test scores at my school will be called X Bar, whose true? Mean is to note that when doing these tests, it’s clear that scale has to matter. For example, I shouldn’t get a different result if my scores are out of 100 or if they are out of one thousand. Well, it turns out that this is nothing but our confidence interval, the way that a statistical test works is like this. Suppose that my null hypothesis is that musical communi and my alternative hypothesis is that Mbewe is not equal communi. Remember that Mbewe is the expected value of my school’s test score. Another way of asking this question is to look at what is the confidence interval for MU? Remember that there’s five percent of probability mass that I want to consider highly improbable. That’s 2.5 percent on the left and 2.5 percent on the right. If this moonshot not falls into either of those regions, either on the extreme left or the extreme right, I will reject the null hypothesis. This area where I can reject the null hypothesis is called the rejection region. Conveniently, I can calculate the boundaries for this rejection region in exactly the same way that I calculated confidence intervals. In fact, if I take X bar, subtract Moonah and divide by the standard deviation in order to normalize the test scores for scale. This is exactly the expression we had before for the confidence interval. As you’ve seen, the rejection region after doing this, standardization is minus one point nine six and plus one point nine six, therefore I first calculate X bar minus MU, not divided by Sigma over the square root of then we’ll call that Z. By the way, this Z is called a test statistic. Then I check whether or not Z is bigger than one point nine six or less than minus one point nine six. If it is, I reject the null hypothesis that Mew is equal to midnight. This test is called the Z test. Using this picture, we can now discuss the meaning of the P value, the p value is just the probability of seeing a result as extreme or more extreme than what we observed. In other words, suppose that I did see a very extreme result and therefore reject the null hypothesis. Suppose that my test statistic lies at minus two. The probability of seeing something as extreme or more extreme than minus two is the probability that the test statistic is minus two or less or plus two or more. So we have to account for the area under the curve in both tails. This is called a two sided test because when our null hypothesis is that musical Ta’amu not the alternative is that BU is not equal to Moonah. That means mu could be greater than not, but also it could be less than Moonah. The way that we calculate this p value is to take one minus the CDF of the absolute value of Z, whereas he is the test statistic and then multiply that by two to account for both sides. We take the absolute value to account for the case where Z is negative. You might want to work this out on paper to confirm that this is correct. A similar test, which is probably of more practical significance in our case, is the one sided test. For example, suppose we want to know whether or not our treatment improves clinical outcomes. This is different from, say, comparing the standardized test scores for my school because of my test scores are worse than the average or better than the average. Either way, I would consider these significant results to bring this back to finance. It’s easy to see how this can apply. Another example for stock returns is that I want to know if my returns are significantly greater than zero. If they are less than zero, I don’t care. I can simply choose a different stock. Now, when you’re testing multiple things at the same time, you have to be careful. But that’s outside the scope of this lecture. In this case, the null hypothesis is that the expected return is less than or equal to zero. The alternative hypothesis is that the expected return is greater than zero. In this case, if our significant threshold is five percent, the entire rejection region is located on a single side of the distribution. The P value will be significant if the test statistic is beyond this threshold to calculate the P value is just one minus the CDF of Z, whereas the is the test statistic. Note that this is half the value of the two sided tests. Therefore, if you have a library that only offers two sided tests, you can simply divide the P value by two. But note that this formula only works if you’re checking the greater than case as your alternative hypothesis. You could read arrive this for the less than case if you wanted. One final note for this lecture is that this was to build your intuition only as you recall, we don’t actually know the true sigma and therefore we have to estimate it using the sample standard deviation. When we do this, our test statistic becomes ti distributed rather than normal. So in actuality, one would use the test rather than the Z test. Furthermore, note that the T test makes the assumption that your data is normally distributed. But again, many people often use the T test as an approximation anyway, just as they do for confidence intervals. Finally, note that we will be discussing all kinds of statistical tests in this course. What we just looked at is a simple example that’s very easy to draw a picture of. Of course, we’re not going to have time to derive them all in the coming lectures. We’ll be looking at test for whether or not our data comes from the normal distribution or any distribution in general. Specifically, the tests can be used for testing normality for this test. The null hypothesis is that the data is normal and the alternative hypothesis is that the data is not normal. So if you reject the null hypothesis, you’re saying that the data does not look like a normal distribution. Another test in Sibai is simply called Normal Test, which does the same thing. Again, the null hypothesis is that the data is normal and the alternative is that it’s not. Finally, we will use the common graph Smirnoff test, which can test for any distribution. Again, the null hypothesis is that the data does come from that distribution and the alternative is that it does not. Finally, I just want to reiterate that in this course, understanding the mechanics behind statistical testing is mostly optional. I could make an entire course on this subject and in fact, I have will be looking at lots of tests in this course, and we will definitely not have time to explain them all in detail. Therefore, it’s sufficient to just understand the API of a hypothesis test as input. We have our data, we have a null hypothesis and we have an alternative hypothesis as output we get a P value and optionally the test statistic associated with that P value, we check the P value against a significant threshold. Typically we’re going to use five percent. If we get a P value below five percent, we’ll say that the result is significant and we will reject the null hypothesis. Otherwise we will fail to reject the null hypothesis.

2. Getting Financial Data

In this lecture, we are going to talk about where you can obtain financial data. The first thing I want to mention in this lecture is the obvious thing. That is, if you already work in finance, then you’ll have access to lots of data through your company. If that applies to you, then you don’t need this lecture. It’s worth noting at this point that because finance is such a competitive industry, data is not free. Some companies collect and curate data, while others pay those companies for their data. If you look at other areas of machine learning, the same kind of idea applies. Sure, you have your amnesty and your image in it and your chest x rays, but these are well-known public data sets. Companies on the bleeding edge of machine learning also have secret repositories of data. For example, you might imagine that companies in the self-driving industry have proprietary data they’ve collected from their own vehicles. Companies making speech to tech software might have their own repository of voice data. For these companies and companies like these, they wouldn’t want to share their data because that’s their competitive advantage. Same thing goes for finance, although generally speaking, most machine learning researchers tend to be pretty open with sharing their data and their code. I want to give a special mention to a company called Quando, which is a very popular resource for financial data, QUANDO has their own API, which can be accessed through simple requests or with language specific libraries like Python, AR and even Excel some parts of quinella free and some are not. Broadly speaking, Canada is split into two parts, which are core financial data and alternative data. Alternative data is for institutional clients only, which is probably not you. Nonetheless, alternative data is probably the most interesting data when you really want to gain an edge. We’ll discuss more about this later in the chorus. So core financial data contains the stuff you would be interested in in this course, such as daily stock prices, and it even contains fundamentals data. Just as a short aside, fundamentals data is data that we are not going to look at in this course, but that could be used in a machine learning context. Fundamentals data includes both quantitative and qualitative data about a company. Information such as profitability, revenue, assets, liabilities and growth potential are considered fundamentals. Some specific numbers that financial analysts use are the debt to equity ratio or ratio and the price to earnings ratio or PE ratio. Since we will not be looking at this kind of data in the course, it won’t be discussed further. In terms of quandongs, other core financial data, you’ll find that it comes in two tiers, free and premium premium data is the data that costs money and each data source is priced separately. So you can pay only for the products that you want. Note that the pricing is on a subscription basis. So you would pay some monthly fee or some yearly fee. If you’re interested in using quando for yourself, this is what I would consider to be a regular programming exercise. Basically, you read the documentation, look at the examples, and as long as you know how to program, it shouldn’t be too difficult. But note that we will not be using Quando in this course. I just wanted to let you know of its existence so that if you want to do your own research in the future, this is one place you could look. Another popular data source is Quand topia. This is not a repository of data per say, but rather it is a platform that allows you to test different trading strategies. Your code actually runs on the platform. So, of course, it ingests data as the BAC tests are being performed. Finally, we’re getting closer to what we will use in this course, you might be surprised to know that Panas itself has a related library for downloading financial data called Panas Data Reader. In fact, the story of Pendas is that it was invented in a financial institution by a developer of the name Wes McKinney. Luckily, he was able to convince the company to open source the library for all of us to use. You’ll notice that as you use Panas more and more, that is quite popular with people who do quantitative finance. And a lot of its functionality is pretty specific to computations you might want to do in an analysis of financial data. So in Panas data reader, you actually have access to multiple data sources, including QUANDO, which we just previously discussed. This is just a wrapper around QUANDO and other APIs. So you still need an API key if you want your request to succeed as before. This is just an exercise in regular programming and reading API documentation, which most of you taking this course will find quite trivial. I’d recommend going through the documentation and checking out what kinds of data you can get access to, if that’s something you are interested in learning about. One of the APIs that Panas connects to is the Alpha Vantage API, which I think is also worth a mention in this lecture. Alpha Vantage is a startup which is backed by Y Combinator, and it has that modern, easy and accessible feel which is more common with young companies today. They provide you with a free API that allows you to download financial data. And if you need to go beyond their free limits, which are quite generous, there is a patent here as well. Their API includes both historical and real time stock prices for an exchange, cryptocurrency and more. Now, pay some attention to the name of this company, you’ll see the word Alpha used a lot in finance. I think this is a nice tie in to some of the concepts we will learn about later in this course. Finally, we have the Yahoo! Finance API, which is the inspiration for a library we will look at briefly and make use of in this cause, I say briefly, because once we’ve downloaded our data and converted it into a CSV, it will no longer be needed. So the Yahoo! Finance API is actually a classic API for financial data, and it’s always been near the top hit on search engines when you search for financial data, which was the case for many years. However, these days it’s pretty well known that Yahoo! Is not the company it once was. And in fact the API was shut down in twenty seventeen. However, there is a Python library called Why Finance that lets us download historical market data without an epic, which makes it convenient for us to use in this course. This library does not wrap Yahoo’s API, but rather it scrapes Yahoo Finance itself. Either way, it works and it’s simple. So it’s a good fit for us.

20. Statistical Testing (Code)

In this lecture, we are going to look at statistical testing in code. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. To start, we’re going to import the drug test and the normal test from Zippi Pirate Stats. These are two functions that test for normality and return to both the test statistic and the P value. You’re encouraged to check out the documentation for these functions if you want to learn more about the arguments and the return values. So first, we call the function Jaque Barer on our values array. If you recall, this array is just an array of Starbucks returns. What we see from this is that the test statistic is very large and accordingly the P value is very small. In fact, it’s so small that it gets rounded down to zero. Therefore, we reject the null hypothesis that this data comes from a normal distribution. Another test we can look at is the normal test, interestingly, this test is not named after a person, but you can check out the documentation. And Zippi, if you want to see which statistician’s inspired this test, if we look at the results, we can see that they’re encapsulated in a normal test result tuple. But the two values it contains are what we expect to see. What we see is that the test statistic is, again, a very large and the P value is, again, very small, about three times 10 to the power, minus 38. Again, we reject the null hypothesis that this data comes from the normal distribution. Not that we can try these on the log returns as well. So in the next few lines, I’m calling the same functions, but I’m calling them directly on the log return series stored in our data frame. Note that we have to drop the net values, otherwise they might be considered part of the data. If we look at the results, we again confirm that the log returns are not normally distributed. Next, we’re going to try the common ground Smirnoff test now, although we could try this with the normal distribution. I think we’re pretty convinced at this point that our data is not normal. So we’re going to use the T distribution instead, since we established that this might be a decent fit. Note that in order for the test to work on an arbitrary distribution, we have to pass in the CDF as an argument. Now recall that Zippi Functions expect its probability functions to not have any extra parameters other than the data. That’s why we had problems earlier with the T distribution, since it has a required parameter for the degrees of freedom. Therefore, we’re going to have to create our own acedia function. But first we need to actually determine the parameters of this t distribution. Luckily, we already know how to do that using the function t that fit. Next, we define our function as follows. It simply calls to CTF, but passing in the degrees of freedom, location and scale parameters that best fit our data. Finally, we call case test passing in our data and our CDV, the result is that we get a very small test statistic in a very large P-value. This indicates that we cannot reject the null hypothesis and in fact, we’re not even close. In the next block of code, we do the same test, but on the log returns again, we obtain a similar result. We cannot reject the null hypothesis that the data comes from the T distribution. Next, we’re going to switch gears a bit and do a one sample t test, so now we are no longer testing what distribution the data comes from, but checking how the data compares to some value. So for this, all we need to do is call the function t test, underscore one stamp passing in the data and the mean for the null hypothesis. For us, that’s going to be zero. If we look at the results, we get a P value of zero point zero eight five. In fact, this is pretty close to our significant threshold, but we still fail to reject the null hypothesis. If we check the same test for the log returns, we get an even higher P value. Now, it’s important to remember that these are two sided tests as an exercise, you might want to try to figure out what the P value should be for the one sided test. Now, because that test didn’t show significance, I wanted to try another stock this time, 3M, which has the ticker. Mm. So first I filter all the rows where the name is equal to Mom and assign that to the variable. Mm. Next we plot the data so you know what it looks like. As you can see, there’s a pretty clear trend upward. Next, I do the usual calculations to calculate the return and the log return. Next, I calculate the mean and standard deviation for the return. Note again how close this is to zero, but remember, this does not imply that they expect to return, it can simply be rounded to zero. The scale in your mind is different from the scale of financial returns. Recall that when we looked at the prices, we saw that they generally trended upwards, whether or not this data is significantly greater than zero depends on the number of data points we have, as well as its variance. Next, we draw a histogram of our returns looking at this, your intuition, and might tell you that this is simply a bell curve centered around zero. That’s why I don’t like intuition. Intuition is wrong, and that’s why we need math. Next, we go through pretty much all the same exercises that we did previously. The next step is to calculate the 95 percent confidence interval. This is the same equation as before, so I’m not going to review it again. The next step is to plot our confidence interval along with the zero line at this point, you now know what the 95 percent confidence interval represents with respect to this zero line. We’ve actually done a test. In this case, the 95 percent confidence interval does not include a zero. Next, we run the test on both the returns and log returns. This time we get a P value of about zero point zero one in both cases, and therefore we reject the null hypothesis that these returns come from a distribution with Trumaine zero.

21. Covariance and Correlation

In this lecture, we are going to discuss the concepts of covariance and correlation. I always like to start my discussion of correlation by looking at the word correlation. As you may already know, correlation describes how related to Renova Variable are. For example, weight is correlated with height. The taller you are, the heavier you are, generally speaking. Another example is that your exam grade is correlated with how much you studied. Generally speaking, the more you study, the better your grade. Many people don’t notice this, but the definition of the word is in the word itself. Correlation is just the combination of two tokens, code and relation. In other words, what is the relation between the two core recurring features? To explain what the point of this is, let’s remember that currently our job is learning how to describe and characterize our data. Correlation helps us do that. For example, we might notice that every time the S&P 500 goes up, so does the Dow Jones Industrial Average. This is called a positive correlation. We might also notice every time that gold goes up, the dollar goes down. This is called a negative correlation at this point. This is pretty much all we can say. What we can do a little bit of foreshadowing correlation and covariance will become very important in later sections of the course. It’s going to pop up again when we talk about TIME series analysis and again when we talk about portfolio optimization. So suffice it to say, it’s an extremely relevant concept in finance, although at this point it may just seem like a mathematical curiosity. I found that this picture is a pretty good description of correlation when the correlation is one, that means there is perfect correlation between two stocks. That means whenever one stock moves up by one dollar, the other stock moves up by, say, exactly three dollars. Whenever the stock moves down by one dollar, the other stock always moves down by exactly three dollars. That describes a perfect linear relationship between the two stocks. Note that the correlation is always between a minus one and plus one. So minus one describes the opposite situation. When one stock goes up, the other goes down by some constant amount. Anything in between describes a more noisy relationship. That is, when one stock goes up by one dollar, the other stock goes up by two dollars. But on another day, maybe the other stock goes down by zero point five dollars. In other words, because the linear relationship is imperfect, the other stock is harder to predict accurately. And that’s what happens when we have a correlation, less than one zero correlation. It means there is absolutely no linear relationship between the stocks. In fact, we will make plots exactly like these between pairs of stocks in order to check how correlated they are. OK, so now that intuition time is over, what does correlation really mean? Well, correlation depends on covariance, so we have to discuss that first, as you may be able to tell by its name, covariance is closely related to variance. As you recall, variance is defined as the expected value of a random variable minus its mean all squared. By the way, if it’s not clear by now, whenever we talk about these statistics, we are usually talking about the return and not the price. So everything in this lecture is in relation to stock returns and not stock prices. All right. But now let’s suppose we are managing our own hedge funds and we have a set of 500 stocks. Now we can ask what’s the covariance between stock return and stock returns? That’s defined as the expected value of r.i minus Mooi times of RJ minus Muj here r.i is the return on Starky and mü is the expected value of the return on stock. What. It should be clear that the covariance between a stock in itself is just the regular variance. From here, we can define the covariance matrix, the covariance matrix is just a matrix that collects all of the possible covariance and variances into a square. The element at Roli and Columb is the covariance between StockTwits and I and stock returns. Furthermore, the elements on the diagonals are the variances for each stock return. Note that if we have these stocks to consider in total, then this matrix is of debugged also because the definition of covariance is symmetric and I and J. The covariance matrix is a symmetric matrix. That is the element at location. IJA is the same as the element at location jieyi. Now, somewhat confusingly, we usually write all the diagonal elements as sigma is squared. That makes sense because this is just the variance. However, for the off diagonal elements, we write them as Sigma IJA without any square. This is despite the fact that the units of Sigma IGY are still squared units. So just keep that in mind. Now, although we will never have to do this calculation and code, it does help to see how one might calculate the sample covariance from a data set consisting of multiple stocks. Let’s suppose I have an end by the matrix of stock returns, that is, I have any samples and the stocks as we typically represent data and statistics and machine learning. I want to calculate the divide covariance matrix for these stock returns. First, I calculate the sample mean of each stock return. This gives me back a vector of length. We’ll call that our bar. Then I take the big R matrix, subtract our bar and multiply that by itself. Then I divide the result by N minus one, which gives me the unbiased estimate of the covariance matrix. Note that I’m assuming that broadcasting is being done so that we can subtract a vector from a matrix. Mathematically, this is incorrect, but it would work in code and this makes it simpler to write down. Note that we can also write this in outer product form, which does not require any broadcasting. This is because our Sabeen is a length vector. It’s just the anthro of the matrix are. Alternatively, you can just calculate each element one at a time by doing the sum over all little N from one up to N of RNA minus R Bahri times orange minus Aubagio. And then you take that sum and divide by N minus one as an exercise. You might want to test this calculation against the co function in numpty. OK, so I promise that the correlation is somehow related to the covariance, in fact, the correlation is just the covariance scaled by the standard deviations of I and J. In other words, the correlation between stock and stock J is the covariance between stock and stock divided by the standard deviation of stock I divided by the standard deviation of stock. J. This also explains why the correlation is always between minus one and plus one. This is a natural result of scaling by the standard deviations. Because of this note that we can also define the correlation matrix, which is just a matrix that collects all of the individual correlations in a table, also, since the covariance between Stockey in itself is just sigma squared, the variance, if we divide that by Sigma II twice, we will always get one. Therefore, the correlation between a stock return and itself is always one that makes sense. There is a perfect correlation between a stock return and itself.

22. Covariance and Correlation (Code)

In this lecture, we are going to look at different ways of analyzing the covariance and correlation of stock returns in code. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Let’s start by downloading the file SP 500 closed at CSFI. As you recall, this is the preprocessed data set where I’ve kept only the close column and have lined up all the stocks by date. Each column is a different close price. Any row is a different day. Next, we load in the data frame using PDF to read CSFI. Now, let’s suppose I want to know the correlation covariance between the following stocks, Apple, Google, IBM and Netflix and Starbucks, the first step is to filter all the columns for these symbols. We’ll call this new data frame sub. Next, we’re going to drop any rows that have all elements as NLM. Since these probably correspond to non-trading days as before we specify access equals zero, which means to drop rows in columns and we see how it goes all so that it only drops rows where all columns are done and we say in place equals true so that the operation is done on our existing data frame. Next, we call the head function to check the resulting data frame. Notice how in this data set, many of the Google prices are missing. If you prefer to build this data set using your own data from Yahoo! Finance, please feel free to do so. Although keep in mind, it doesn’t affect the exercise. Next, we’re going to calculate the returns for each symbol. Notice how the new columns are simply the symbol concatenated with underscore priv and underscore red. Also note that we didn’t have to do this calculation manually, although there is generally no harm in doing so. Next, we create a new data frame, filtering out only the columns that contain the returns that we just created. We call the head function again to check on new data frame. As expected, the first row is Enan. The next step is to plot what is called a scatter matrix, Seabourne has a nice function to do this called Purply. So we’ve imported Seabourne and called the pair plot function, passing in the returns data frame. Basically, a scatter matrix works like this. Both the rows and columns correspond to the returns where each stock on the diagonals we have the histogram for each return on the off diagonals, we have a scatterplot of each return versus each other return. So for example, Apple versus Google, Apple versus IBM, Google versus Starbucks and so forth. Basically it’s a visual representation of the covariance matrix or the correlation matrix. We can see directly from this plot how correlated each of the returns is recalled from our visualization earlier, that a perfectly straight line would be a perfect linear correlation, something that looks like a circular cloud would mean no correlation, and everything else is in between. So from this plot, we can see that there is a decent amount of linear correlation between Apple and Google, so Apple is number one and Google’s number two. So this box right here is Apple versus Google. And you can see there’s a bit of linear correlation. We can also see a decent amount of linear correlation between Google and Starbucks and Google and Netflix. So this one’s Google and Netflix, this one’s Google and Starbucks. And for the others, I don’t think the correlation is very clear. Next, let’s calculate a few more basic statistics for each return. First, we have the mean that’s red stop mean and we have the standard deviation that’s red state as TD. The next thing I’m going to do is plot the mean returns against the standard deviations in finance, the standard deviation is one way to think of volatility. This plot helps us see a pretty common phenomenon in finance. We see that this plot generally trends upwards. That is when the volatility is higher, the return is also higher. Finally, let’s calculate the correlation matrix and the covariance matrix. Luckily, these are just single function calls. So first we have rescore these results seem to confirm our suspicions from the scatter matrix. The correlation between Google and Apple is pretty high above zero point for the correlation between Google and Netflix is also high, also above a zero point four. And the correlation between a Google and Starbucks is the highest above zero point four or five. Next, we do the same thing for the covariance, which is just red Pskov. As expected, these values are very small. Remember that we’re taking the standard deviations which we saw earlier and squaring them. So we expect these values to be pretty tiny because of this. The correlation matrix is slightly more useful because it means all the values are normalized to be between a minus one and plus one.

23. Alpha and Beta

In this lecture, we are going to discuss a fundamental concept in finance, alpha and beta. Unfortunately, many resources from the concept of alpha and beta as mysterious and overly mathematical. And in fact, as you’ll see, alpha and beta derive from one of the simplest ideas in machine learning. There’s nothing mysterious about them at all. So there is a good reason why this lecture goes right after our discussion on correlation. As you recall, when two stocks are perfectly correlated, a scatterplot of the returns will show a perfectly straight line. They can either be perfectly correlated or perfectly a. correlated. That just means the line will go up into the right or down into the right. For students of machine learning and statistics, the first algorithm we always learn about is linear regression. Another name for this is the line of best fit. This is a model that we use when we observe a linear correlation between some input and some output, kind of like how we do right now. The line serves as a predictive model. That is, I can ask a question like if I work for three years, what will be my salary? If I work for four years, what will be my salary? In fact, many students learn about this technique in high school, but without being told that they are doing statistics in grade school, you usually take a ruler and try to find the line of best fit visually. Of course, as you progress to college and beyond, we use calculus and maximum likelihood estimation in order to find this line more precisely. Since this course is not about linear regression, I’m not going to bore you with the details about how the algorithm works, but rather I’m going to focus on the modeling aspect. In grade school, we learn that a line is expressed as Y equals M, X plus B in this form, X represents the input variable which goes along the horizontal axis. Y represents the output variable which goes along the vertical axis. M represents the slope, which tells us how steep the line is, and B is the Y intercept, which tells us where the line crosses the Y axis. In other words, B is the value of Y when X is zero. Let’s see how this applies in finance. Well, you’ve already seen that we can draw a scatterplot between the returns of pairs of stocks. Most typically we put the return of the market on the x axis and the return of some other asset on the Y axis, for example. Maybe we want to compare Apple’s stock return with the market or the stock return on some retirement fund with the market. Another option is to compare one stock with another stock. But let’s assume for this lecture that we want to compare against the market. Typically, it’s conventional to assume that the market is represented by the S&P 500. As you recall, the symbol for this is SPI. Now, let’s say we take the stock returns for Apple, we take the stock returns for S.P.I, we create the scatterplot and then we draw the line of best fit. Well, it turns out that alpha and beta are nothing but the intercept and slope of this line. That is beta tells us how much Apple moves relative to the market. If beta equals two, for example, and the market goes up by one percent, then Apple is predicted to go up by two percent. If the market goes down by five percent, then Apple is predicted to go down by 10 percent. On the other hand, Alpha is how much better Apple is irrespective of the market. For example, an alpha of one percent means that the return for Apple was one percent better than the market. For the period in which we looked at the returns, a positive alpha means Apple has performed better than the market. A negative alpha means Apple has underperformed the market. Conceptually, we want to achieve high alpha because we want to outperform the market. So how does this work in code, as mentioned in this class, we won’t be overly concerned with implementing machine learning models unless it’s absolutely necessary. So we will turn to our old friend Saikat, learn in general so I could learn. Models are designed for vector observations. That is, you would try to predict an output y based on multiple axes. And example of that would be something like AASA by the return on assets ie is equal to one times R one plus beta two times R2 plus beta, three times are three plus alpha. So you have three different returns as your predictors rather than just the one return, which is the market. In fact, this is a more advanced type of model in finance known as a factor model. This will probably be outside the scope of this cause. In any case, the point of this is to say that we can’t simply pass in a one dimensional array of returns since in Saikat learn, our data is expected to have shape and biddy’s where and is the number of samples and is the number of features in our case and as the number of days for which we have stock returns. And there’s just one because we’re comparing to just the market. So in order to use circular and for linear regression, we need to make our array of returns to the array of size and by one rather than a one to array of size n. After that, we use the Psychic Learn API as usual, first we create an instance of the model, second we call model that fit X. In this case, X is the returns on the market. And why is the returns on whatever stock we are comparing to the market? Third, if we want to make predictions or draw the line of best fit after fitting the model we call model predict X. To summarize this lecture, we just learned about the fundamental financial concepts of alpha and beta, we learn that these are nothing but the parameters of a line of best fit alpha is how much better or worse an asset performs compared to the market. And Beta tells us how much an asset moves relative to the market. Now it makes sense why companies often name themselves after the term Alpha. We saw an example of that earlier in the course with Alpha Vantage, a data platform. Another example is Bridgewater, Pure Alpha, a famous hedge fund. Note that these say nothing about how accurate the model is or how noisy the data around the line is. For that, we would need to look at the correlation in the next lecture. We look at how to find alpha and beta encode.

24. Alpha and Beta (Code)

In this lecture, we are going to look at how to calculate alpha and beta in code as usual, you can look at the title of the notebook to determine what notebook we are currently looking at. To start, we’re going to download our data. We need both the spy data set as well as a subset of the S&P 500 so we can compare one of these stocks to spy. Next, we’re going to import our usual libraries numbed by pandas and matplotlib. Next, we’re going to load in the ASPI data set, as we normally do, using pedigreed CSFI. Next, we’re going to calculate the return for spy using the percent change function. Note that I’m giving this column the name spy instead of something generic like return. This is because I’m going to join this with another data frame later. So I want it to have a more informative name. Next, I’m going to load in the S&P 500 stocks using pedigreed CSFI, I’m going to call this data frame index since it contains stocks from the S&P 500 index. Next, I’m going to extract the Apple stocks by choosing only the rows where name equals Apple. Next, I’m going to calculate the returns for the Apple stock by using the percent change function again, notice again how I’m giving this a unique name by calling it Apple instead of something generic like return. Next, I’m going to create a new data frame called Joint. This is going to equal the Apple data frame filtering only the return column joins with the spy returns. Next, I call the head function to ensure that my new data frame is in the correct format. Next, I’m going to plot this data frame, which means it’s just going to plot every column as a time series as usual. Note that since returns are very noisy, it will be easier to see a clear pattern if we select a small subset of the data. So I’ve used I look to select the rows from 100 to 150. Obviously, this is completely arbitrary. You can choose whatever rows you feel like. Now notice from this plot how closely apples returns track the market returns. And in fact, they also appear to be on the same scale as well. So when spy goes down by this much, apple goes down about the same amount. And we see the same pattern here, so you don’t see Apple going much higher than S.P.I or much lower than S.P.I. And based on this, try to think about what Baner might be, will it have a large value or a small value? Will it be positive or negative? Next, I’m going to create a scatterplot of Apple vs. Spy. Notice that from this plot, the correlation doesn’t appear to be as strong as it looks in the above plot. It does look like there is a clear upward trend. Next, we’re going to work on creating our data set so that we can do some machine learning, although we don’t need to convert our data into an umpire. We’re going to do it anyway, since I think it’s easier to understand that machine learning works on numbers and not data frames per say. So the first thing I’m going to do is drop all the names. We don’t need those when we’re doing machine learning. Next, we’re going to filter out S.P.I in April. So by passing in a list of column names, even though each list only contains a single symbol, we get back a data frame, which is a two dimensional object. Then we call the two numpad function to convert the data into no arrays will assign these to the variables X and Y. Next, when we check the shape of X and Y, we see that they are in fact, and by one two dimensional arrays. And remember, that’s what we need for machine learning. Not that it’s not necessary for Y to be two dimensional, but there’s no harm in making it two dimensional. Next, we demonstrate that if we do a scatterplot of X and Y, we get the same plot as before. So nothing has changed. The data is just in a different format. Next, we import the linear regression class from Saikat learn, then we go through the usual steps of instantiating a model and calling the fit function on the data. By the way, if I haven’t reminded you yet that I teach all about how to use, I could learn in my free course and you don’t know how to use it, then you might want to go and check that out. If you find this confusing. As always, since I teach this stuff for free, I don’t anticipate any problems with learning these basics. In the next block, we’re going to draw the scatterplot again, along with the line of best fit in order to draw the line, I’m going to first create an input array X predict of equally spaced points. Note that because this is a line, we really only need two points, one of the beginning and one at the end. I just chose five points arbitrarily. As usual, we go from the Minox to the max X so that the line spans across the entire data set. Also note that I reshape the line space output to end by one so that this can be passed into the socket learn model. Next I call model that predicts and this gives me y predict and next I draw the scatterplot again along with the line from the data I just created. So what do we see? Well, it’s pretty much what we expect. There’s our data and there’s our line of best fit. Next, we’re going to retrieve Alpha and beta from our fitted model. You might want to check the cyclone documentation so you can learn about the models attributes. Specifically, the slope is located in the attribute coif and the intercept is located at the attribute intercept. As we discussed in the theory lecturer in finance, we call these beta and Alpha. All right. So what do we see? Well, we see that Alpha is very slightly positive. This is about four point six basis points and beta is approximately one. That makes sense because in our plot earlier we saw that Appel’s returns lined up very closely with SPI. Next, I’m going to show you how to draw the exact same plot we had previously with the line of best fit, but this time we’re going to calculate the line manually using alpha and beta. This will prove to you that the equation I described to you before does, in fact, correspond to the line that is beta times the market return plus alpha. All right. So we see that our new plot looks exactly the same as our old plot that’s confirming that our equation is correct. Now, you might think perhaps it’s better to invest in Apple than it is to invest in the market due to its positive alpha. However, another important aspect of finance is risk or volatility. As mentioned previously, one common way to measure this is with the standard deviation. If we look at the standard deviation of our data frame, we see that the volatility for Apple is about twice that of the market. So while Apple did have better returns, it also had higher volatility. This is a common pattern in finance where, generally speaking, if you want higher reward, you have to be willing to take on more risk.

25. Mixture of Gaussians

In this lecture, we are going to explore yet another machine learning model, this time from unsupervised learning. As you recall, one of our main tasks in this section is that of density estimation. That’s just a fancy way of saying we’re trying to figure out the probability distribution of the returns. So far, we’ve discovered that the returns are heavy tailed and that a single Gaussian is not good enough to capture this phenomenon. We then looked at the T distribution as a possible solution. In this lecture, we will return to the Gaussian. This lecture is all about a technique called mixture of Gaussians or the Gaussian mixture model. You can tell it’s a technique from machine learning because it uses the term Gaussian rather than the term normal. As you recall, these both refer to the same kind of distribution. In this lecture, I’m going to first describe what the Gaussian mixture is and how it works, then we’ll discuss why it works for modeling returns. So what is a Gaussian mixture? Although we’ll be using the Gaussian mixture for density estimation, it can also be understood in terms of clustering. As you may recall, clustering is an unsupervised machine learning technique. The basic idea is this. Suppose I have a data set. Perhaps I’ve gathered data on all the students in my class. They are test scores on several tests. Let’s suppose my class has two tests so I can plot these test scores on a two dimensional scatterplot. When I look at this plot, I’ll be able to see some patterns. For example, in the upper right corner, I’ll see students who performs well on both tests. In the bottom left corner, I’ll see students who performs poorly on both tests. In the upper left corner, I’ll see students who performs poorly on test one but performs well on test two in the bottom right corner. I’ll see students who performed well on test one, but poorly on test to these clusters might help me decide how to best assist my students in studying for their final exam. Perhaps the students in the upper right corner don’t need any assistance. They are doing just fine. Perhaps the students in the bottom left corner need the most assistance. They haven’t performed well on any test. Perhaps the students in the upper left corner slacked off at the beginning of the course, but now they are on track to ace the exam and perhaps the students in the bottom right corner started off doing well. But some life event caused them to not study as hard. Later in the chorus. So what does a Gaussian mixture model do? Well, the Gaussian mixture model assumes that each of these clusters takes on the shape of a Gaussian. As you can see, this allows for a much more expressive estimate of the density of the data. When you see multiple homes like this, it’s called a multimodal distribution. It’s clear that a single Gaussian or a single T would not be able to capture these details in terms of clustering. We can take a data point and ask our Gaussian mixture model. Which of these Gaussians does that data point belong to? So using a concrete example, I can take my students to test scores, plug them into my model and my model will tell me, does the student belong in cluster one or two, three or four? So what does a Gaussian mixture model look like mathematically? In fact, it’s just the way that some of individual Gaussian pdf we usually denote these weights with the symbol PI. Note that all the PIs have to sum to one since in order to be a valid distribution, the entire thing must integrate to what we already know that each individual Gaussian integrates to one. And therefore it’s easy to show algebraically that all the pies must also sum to one. At this point, you might ask, how do we know what each of the PIs are and furthermore, how do we know what the means and variances of each Gaussian should be? Finding these parameters is called learning. So in machine learning, the act of finding these parameters is what learning entails. Luckily, the math behind the learning algorithm is quite complicated. And again, that itself takes an entire course to discuss. So it will be considered outside the scope of this course. If you’re interested, basically just know that it’s the same old story. What we want to do is maximum likelihood estimation. That is, we want to maximize the likelihood of the data with respect to the model parameters. We can accomplish this by using algorithms such as expectation maximization. In code, it’s a simpler story, we can use the Cyclone API with all the usual functions. First, we create an instance of the model, then we fit the model to the data by calling the function. Note that because this is unsupervised learning, there is no target and the fit function only accepts as input the Data X.. If you want to know which cluster a set of data points X belongs to, you can call the predictive function which will return those cluster identities. At this point, you must be wondering what is the point of using Agastya Mister, which is clearly for multimodal distributions on stock returns, which have not looked multimodal so far? Well, this may be surprising, but the purpose of using the Gaussian mixture is not for its ability to model multimodal distributions. Instead, it’s for a less well known property, which is that Gaussian mixture is have heavier tails than single Gaussians. In fact, it’s possible to demonstrate this algebraically, pick any Gaussian with arbitrary mean mu and variance sigma squared. Then suppose I have a Gaussian mixture. Pi times Gaussian one plus one minus pi times Gaussian to Gaussian one has me mu and variance sigma one square Gaussian two has been mu and variance sigma two square sigma one and sigma two can be set such that the variance of the mixture model overall has the same variance as the original sigma squared. It’s possible to show that the courtesies of the Gaussian mixture is larger than the courtesies of the single Gaussian. For us, we’ll simply demonstrate numerically that this is the case using Python. Although as an exercise you might want to try this yourself on paper.

26. Mixture of Gaussians (Code)

In this lecture, we will look at how to apply the Gaussian mixture model in code. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. To begin this lecture, we’re not going to use the psyche to learn GM right away, but rather we’re going to just create our own test. GMAR and Checketts keratosis. This should give you some intuition for how the GMs can in fact achieve heavy tails, which might seem surprising. We’ll start by creating 500 data points evenly spaced between minus zero point one and plus zero point one. We’ll call that X list. Next, we generate the PDF for an arbitrary mixture model. Note that I’ve chosen these numbers randomly, so my mixture of proportion will be zero point five. I assign that to the variable P. Next I set to be P times one Gaussian pdf plus one minus P times a different Gaussian pdf. I’ve chosen these Gaussians to have the same mean but different variance. So this one has standard deviation zero point zero one, this one a standard deviation zero point zero zero two. Next, I plot the PDF using the plot function, as you can see, this has the characteristics we saw in our returns. It has a very tall head, skinny shoulders, but unfortunately, the tails are hard to see. However, we can easily check the keratosis, so in this next block of code, we’re going to generate samples from our made up distribution and then we’re going to check the sample keratosis of those samples. We’ll start by creating an empty list called samples. Next, we create variables for the parameters of our Gaussians M zero zero and one. And as one, you can double check that these are the same parameters as above. Next, we enter a loop inside the loop. We generate a random number between zero and one we check of. The random number is less than P. If it is, we generate a random number using Gaussian zero. Otherwise we use Gaussian one. You should be able to confirm to yourself that this uses Gaussian zero with probability P and Gaussian one with probability one minus P. So we choose Gaussian zero percent of the time and we choose that Gaussian one one minus eight percent of the time. Inside the statements, we used the Norm ARV’s function to generate a random sample from the normal with the given parameters, when we’re outside the loop, we convert our list of samples into a panda series so that we can easily use the courtesies function. Finally, we call the keratosis function. As you can see, we get a number bigger than zero, which suggests that our samples have heavy tails relative to the normal. Next, we’re going to use a Gaussian mixture model to fit a distribution to actual stock returns. We’ll start by taking the returns from our Starbucks data set. We drop any missing values, convert the result to an umpire, and then reshape the array so that it’s two dimensional. Remember, we want something of shape and biddy’s when we’re using Saika learn. Next, we instantiate a Gaussian mixture object. One downside to the gas mixture model is that you have to choose the number of Gaussians since two is enough to generate heavy tails will use to. Next, we’ll call model Duffett to fit the data. Now, at this point, in order to actually draw the PDF of our fitted distribution, we need the parameters of our fitted model. Luckily, I’ve already looked at the documentation so I know which attributes I need. You may want to check out the documentation yourself to confirm that what I’ve done is right. First, we need the mixture components which are stored in model that. Next, we need the mean of each Gaussian which are stored in a model that means lastly, we need the covariance is which are stored in model dot covariance is now you’ll notice that the shapes are kind of strange. It looks like the means and the covariance is have a lot of extra dimensions. The reason for that is remember that Saikat learned models work with multi-dimensional data. Our returns are just one dimensional. In general, your data will have shape and by D or D the number of features. So each mean vector will be a vector of size D and if you have K Gaussians then all of your means together will be of size combined. Furthermore, each covariance matrix will be of size divide. So if you have K Gaussians then all of your variances together will be combined. Bid for us since our D equals one on our K equals two. The means are two by one and the covariance is are two by one by one, which are actually just the variances. There are no covariance is since we only have a single dimension. Next, we flatten the means and the covariance is to convert them into one dimensional arrays since the one dimensions are redundant. Next, we begin to generate the data for our plot. First, we create a list of exes from the mean value to the max value. Next, we create the PDF for the first ghasem, which I’ll call zero. Then we create the PDF for the second Garcia, which I’ll call eFax one. Finally, we create the PDF for the full model by weighting the two Gaussians by the weights we got earlier. And remember, these are all parameters from the fitted model. Finally, we draw a histogram of our returns along with our fitted density, as you can see, the Gaussian mixture fits much better than just a single Gaussian that captures the tall head, the skinny shoulders and the fat tails.

27. Volatility Clustering

In the previous lectures, we looked at how stock returns might be modeled not just by a single Gaussian but multiple Gaussians, in this lecture, we will begin to explore the idea that, in fact, there might be a time component to stock returns as well. Typically, when we model some random variable as a Gaussian, we think of each sample as being independent and identically distributed. But if there were a time dependent’s that suggests that perhaps some characteristics of the stock return are predictable. So what is volatility clustering? Volatility clustering is the phenomenon we observe and stock returns where high volatility returns are clustered near other high volatility returns and a similarly low volatility, returns are clustered near other low volatility returns. That is to say, if we’ve been seeing high volatility recently, it suggests that we will continue to see high volatility if we’ve been seeing low volatility recently. It suggests that we will continue to see low volatility. And by the way, just so we’re clear, let’s assume that we measure volatility by the standard deviation of the return. At this point, we won’t learn about any machine learning methods that will model volatility clustering. But the point right now is just to observe the phenomenon itself. So if we switch over to the code, we can observe volatility clustering simply by plotting the returns as a time series. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So for this example, we’re going to look at the same stocks we’ve been looking at for the past few lectures Apple, Google, IBM, Starbucks and Netflix. Basically, this code, just as a follow up through all the symbols and then we use the subplot function to organize our plot as a grid. All right. So what do we see? Well, we can observe volatility, clustering. So if we look at Apple, we see some high volatility here, maybe over here, over here. Looking at Google, here are some high volatility, here’s an area of low volatility. Here’s an area of high volatility for IBM. And for Netflix, here’s some high volatility, so basically you’re going to see clusters of high volatility and low volatility, high volatility follows other high volatility and low volatility follows other low volatility. All right, so that was a short lecture on what volatility clustering is at this point. Maybe you’d like to think about ways in which to model this phenomenon using machine learning techniques you’ve learned in the past.

28. Price Simulation

In this lecture, we are going to discuss how to simulate stock prices, although many beginners tend to be overly enthusiastic about trying to accurately predict future stock prices, price simulation is actually much more useful. Simulation helps you understand how your algorithm will behave under uncertainty. You can’t only test your algorithm on historical data. You want to be confident in knowing that your algorithm works. You want to be able to say, my algorithm is robust and my algorithm doesn’t lose lots of money when the market is bad. This lecture will focus on the basics of how to simulate the evolution of a stock price. The best way to understand this is with pseudocode, we’re going to look at two methods of simulating stock prices. The first method is to take all the historical returns from a given stock and then sample from those returns in order to generate future stock prices. The pseudocode goes like this. First, let’s assume that we have an array of returns that we’ve just calculated from our data. Recall that you already know how to do this. Next, we start with some arbitrary start price. We can just set it to be the final close price in our data set. Next, we enter a loop inside the loop. We sample a return from our list of returns. Then we generate the next day’s price by taking the previous price and multiplying it by one. Plus the return that we just sampled. You should be able to confirm, based on our earlier discussions, that this formula is correct. Finally, we save this price to our list of prices at the end of the loop. Our list of prices represents one possible future progression of the stock. Another way to do a price simulation is this, as you recall, it’s possible to fit a distribution to the returns. So far we’ve seen that the T distribution and the Garcia mixture works pretty well. Let’s assume we fit a T distribution. This gives us back the parameters, D.F. location and scale. Again, we set the initial price in our simulation to be the final close price in our data set. Then we enter a loop inside the loop we sampled from our fitted distribution. This gives us a sample return. Then we use the same formula we had earlier to find the next day’s price using this latest return. Again, we save this price to our list of prices and at the end we have one possible future progression of our stock. So you can see that the only difference between this algorithm and the previous algorithm is how we sample the returns. One way is to use actual historical returns, and another way is to fit a distribution and then sample from that distribution.

29. Price Simulation (Code)

In this lecture, we are going to look at how to simulate stock prices in code. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. All right. So the first algorithm we’re going to look at is the one where we sample from the existing returns. First, we set the initial price of P0 to be the final close price of our data set. Next, we create a list of prices. Currently, this contains only P0. Next, we create a list of returns by taking the Starbucks returns column and dropping the missing values. Next, we enter a loop. I have assume that we want to simulate over 100 days, but obviously those choices arbitrary. Next, we use the NPR Random Choice function to randomly select a return from our set of returns, we’ll call this hour. Next, we find the next price by taking the last price in our prices list and multiplying that by one plus R, because this P. Finally, we append P to our list of prices. When we’re outside the loop, we plot the prices as a time series. So you can look at this and decide whether or not this seems like a reasonable and realistic evolution of the stock price. Next, we do a simulation using a fitted t distribution, since you already know how to fit a T distribution. I’m sure this is not too surprising. We call Taffet and this returns D.F. location and scale. Next, we grab P0 and instantiate our prices list as we did previously. Next, we enter a loop that goes for one hundred steps inside the loop. We sample from our T distribution. This is accomplished by calling the ARV’s function. The return value is a sample from the distribution, which is a simulated stock return. Next, we calculate the next price by taking the last price and multiplying by one plus R as before we save this to our prices list and then we plot the final prices. All right, so I think this looks reasonable, I think if I were to show you the two plots side by side, you wouldn’t be able to tell which one it came from, which algorithm.

3. Getting Financial Data (Code)

In this lecture, we are going to walk through some rough code that I wrote in order to download some of the data that we’ll be using in this course. So I just want to remind you that when you see code in a plain text editor, that means I’m looking at code from the code repository. If I’m looking at code in a CoLab notebook, that means I’m not in the course repository. So just keep that in mind if you ever get lost looking for code. I also want to remind you that we’ll be working mostly with CSV files in this course. So if you don’t care about how we got the data, note that this code is not necessary to understand the rest of this course since we’ll be just using Carvey’s. OK, so first of all, we want to look at is called Get DataDot Pie first, we start by importing, we finance and Ossy note that if you don’t yet have Y finance installed, you can install it the usual way, using PIP by using the command pip install WI Finance. Next, we have a list of stock tickers for companies in the S&P 500. I got this list from the Kagle repository, which I will show you later. However, not all of these companies could be found, by the way, finance API. This is not a big issue for us since we’ll only ever look at a few stocks at any single moment in this cause. Next, we append the symbol S.P.I, which is the symbol for the S&P 500 index. Next, I create a folder called Data, which is where we will store our downloaded data. Next, we have a loop that looks at all the symbols and symbols list one by one. Inside this loop, we check whether or not a CSFI file already exists for our symbol in the data folder. If it doesn’t, then we proceed. Next, we call the function we have to download to download stock data for the current symbol of used an arbitrary start date of January 1st, 2010, and an arbitrary end date of December 31st. Twenty eighteen. This is mostly arbitrary. Twenty eighteen was a long time ago at this point, but I don’t think including more data would have been useful for anything that we are going to do. A few dates I did want to capture with a Google and Apple stock splits, which both happened in 2014. You’ll see why those events are important later on in the section. Now, recall I said earlier that not all of the symbols above could be found, by the way, finance API. So when we get back our data frame, we want to check it size. If the size is greater than zero, then we save the data frame in the second loop. In the script I live through, XVII saved and delete any file which has less than 10 lines. Now, obviously I could have just combined these two loops together. However, this is a very rough script and I never actually ran this code as it is. I actually use the Python repo, so I was copying and pasting code around. So if you want to modify this to suit your needs, you can feel free to do so. The next five we’re going to look at is called a pendant pie. You might wonder why did I previously save each data frame in a separate ESV rather than just combining them all into the same CSFI? Well, the reason is, as you’ll see later, they all have the exact same column names and dates. Therefore, we wouldn’t want to combine the curves at that point and be confused about which rows belong to which stock. What we would like to do before combining the data is to have a name column that tells us which stock that row of data refers to, and that’s what this script does. So first we import pandas and the Glub function. Next we get all the file papers for all the curves in the data folder. Next, I instantiate a variable called Foldit to none, and this is the variable that will store our data frame as we accumulate data from each of the Carvey’s. Next, we enter a loop to loop through all the spheres. Inside this loop, we load in the currency ASV by calling PDK CSFI. Next, we determine which simbel this ESV refers to by parsing the symbol from the filename. To do this, we first split the file path on the slash, which is used to remove the folder name, which is data. Next, we split on the dot to remove the file extension, which is CSFI. Next, we create a new column called Name, and we assign it the value symbol, which we just obtained. This will assign the same value to every row in the current data frame. Next, we check whether full D.F. is not if it is, then we can just assign full deserve to be the current data frame. Otherwise we call the appen function passing in the current data frame D.F. and assign the result to Foldit. If we pass in the argument, ignore index, since every data frame we load in will have the same index, which is zero up to however many rows there are. Finally, we save the full data frame, Tursi, as we called ESP 500 full ESV. Now, what we just did is probably overkill, since most likely we will not need all of the nearly 500 stocks that we just downloaded in order to make things more manageable. We have another similar script called a the small dot pi. In this file, we import the random module and we repeat our big list of S&P 500 stock tickers. Next, we have a set called Small Symbol’s, which consists of a few stocks that I’ll be using in this course, along with a few others from different industries. The reason I wanted to do this is because clearly I’m biased as a tech person, it’s kind of a given that I’m going to choose things like Apple, Google, IBM and Starbucks. So the point of this is to choose a few stocks from other industries that are hopefully uncorrelated with the stocks that I was predisposed to choose. By the way, you’ll learn why being uncorrelated is useful later on in this course. So anyway, since I do want a few more stocks to work with, with the help of the random module, I sample one hundred more stocks from our symbol’s list. If there are duplicates with our small symbol’s list, they will disappear in the next step. So the next step is to convert our sample into a set and then do a union of our two sets. As you know, sets can only have unique items, so any duplicates are now gone. Next, we have a loop which stacks all of the data frames from our chosen stocks together in exactly the same way we discussed previously. This time, we save the final data frame to a file called ESP 500, a sub dot CSFI. All right, so that’s how I generated the data that we’ll be using in this course. Thanks for listening and I’ll see you in the next Lechter.

30. Financial Basics Section Summary

In this lecture, we are going to summarize everything we learn in the section, to be quite honest. This section became a lot longer than I had originally anticipated, but this just goes to show how much there is to learn about the basics of finance. If you think you’re going to become a billionaire using an LSM to predict tomorrow’s stock prices, you would have already made one in the time it took you to get through this section in this section. Our goal was to understand the central measurement we want to model in finance, which unbeknownst to many is actually the stock return and not the stock price. This section was all about how to model the stock return. In fact, even after this section, we will continue to discuss how one might model the stock return. We started by looking at some extremely basic questions like where does one download financial data and what is financial data look like? This section is all about how to model the stock return as a non time varying random variable. We investigated whether or not the stock return might be modeled by a normal distribution. We found that based on the Cucu plot and several statistical tests, that this is probably not the case. We then looked at the T distribution, which seemed to work better. The T distribution is a fundamental distribution in statistics as it forms the basis of the T-test and the confidence interval. We looked at several quantities of interest when it comes to stock returns, including the Skewness, keratosis correlation and covariance. We looked at the meaning of alpha and beta, two very popular words in finance, this was our first application of machine learning. We looked at how aggressive Mr. Model might be used to model stock returns. This was our second application of machine learning. We then looked at volatility clustering. This might suggest that instead of a static model like a Gaussian mixture, perhaps a time varying model might be even better. Finally, we looked at how to simulate stock prices, which is very important for testing the effectiveness and robustness of an algorithm.

31. Suggestion Box

In this short lecture, I want to tell you about my suggestion box, the idea behind this is analogous to the kind of suggestion box you would find at a restaurant. You can write down your thoughts and leave them for the owner to look at to give them feedback so they can improve your dining experience. In order to share your thoughts with me in my suggestion box, please go to lazy programmer me suggestions here. You will find a simple form where you can tell me anything you like. I have a few suggested fields, but none of them are required. However, they would be helpful for me to know. For example, tell me your background. I think someone coming from a background of marketing is going to have very different feedback compared to someone coming from a Ph.D. in physics. Tell me what course you’re taking so I have some idea of what you’re talking about, obviously the more specific you are, the better I can help you. Tell me how difficult you thought the course was. Was it too easy? Was it too hard? Tell me if there was something I neglected to explain. For example, was there some word I use that you’d never heard of? Was there a python code that you haven’t seen before? Tell me what you thought was missing from the course, even if it was just something small, like you missed explaining this line of code. Were you looking for an algorithm that wasn’t included? This is helpful for me, but it’s also helpful for you because many times students are looking for an algorithm or an explanation in the wrong course. So I can say actually, I teach that algorithm or I explain that thing in such and such. Course, this isn’t the right course. So have a look over there. Are there any topics you want to request that were not included in the course? For example, maybe you signed up for a deep learning course and you were looking for signs, but the course didn’t have CNN in it. Let me know. Finally, let me know your suggestions for future courses, if there’s a particular topic you want me to teach that I don’t yet have a course about, such as grading and boosting or transformers or even quantum mechanics. Let me know. Finally, there is a big text box for any comment you want to write. You can write anything you want in this box and make it as long as you want. In fact, the longer the better. If you think I talk to you slower, I talk to you fast. Let me know if you think my prerequisite instructions aren’t fair. Let me know. I want you to be as specific and as detailed as possible. A lot of the time people give me comments that I can’t really act on, which is unfortunate. Tell me exactly what lecture and exactly what time stamp you’re referring to. If you want to refer to a question on the Q&A, include the link. Be very specific and give me concrete examples so that I know exactly how to act on the information you’ve given me. Thanks for listening and I’ll see you in the next lecture.

4. Understanding Financial Data

In this lecture, I want to discuss how to understand financial data, the kind of financial data we’ll be looking at is stock price data, along with volume information. If you’ve never studied finance before, you may have assumed that there is just a single stock price, the stock price of a stock. However, in practice, stock price data is not represented that way. Instead, data is represented in this tabular for we have the columns open, high, low, close in volume. Sometimes we also have a column called Adjusted Close, which we will discuss later. Note that for the rows, each row corresponds to a day, but this does not necessarily have to be the case. You can also have weekly data, monthly data or intra day data. Most commonly, however, in the study of financial analysis and financial engineering, we start by working with daily data. So what do these columns mean? Well, first, we recognize that no matter what the frequency of our data is, it represents a block of time and within that block of time, the stock price will fluctuate. So the open, high, low, close columns are meant to give us information about how the stock price changed during that block of time. I think it’s pretty intuitive. The open price is the price at the beginning of the period. The close price is the price. At the end of the period, the high price is the maximum price during that period and the low price is the minimum price during that period. The volume is the total number of shares that were traded during that time period traded. That means that they were sold by someone and bought by someone else. If you want, it may be helpful to think of the stock market as a physical market that people can go to to purchase various items. Anybody who goes to the market can buy and sell from anyone else. So, for example, you go to the market and you carry a bag full of apples, Starbucks coffees and some 3M and ninety five mass, you might think to yourself due to current world events, I believe three a.m. about to go up in value. So you might want to buy more in hopes of selling them at a later date for profit. So you might look over and find Bob who wants to sell his 95 mass, so let’s say Bob is willing to sell 10 mass or five dollars each and you’re willing to buy them for that price. So you pay Bob 50 dollars and in return, you now own 10 more and ninety five bars. This kind of exchange is happening many times per second and the price of each item can change depending on what people are willing to buy and sell them for. In reality, these exchanges are done by a computer and each transaction is stored in what is called an order book, each event called an order, means that someone has offered to buy or sell a stock at some price. In fact, it works more like an auction rather than a direct sale you offer to buy or sell at a certain price. But that is not guarantee that any transaction will actually take place. Any transaction requires that someone is willing to buy at a price at or above the price that someone else is willing to sell. So the exchange collects these orders and matches them so that they can actually be executed. Sometimes the number of shares don’t match up and so some orders will only be partially filled. For example, you might try to purchase ten shares at a certain price, but only five were available. So in the end, you end up only buying five shares. What is interesting is that the data we typically work with in finance, if we consider what actually happens in the stock market, what we actually have is more like a summary of those events. So you might wonder, is any useful information being lost in this process? That’s the question which is outside the scope of this course, but something worth pondering on your own. One convenient aspect of such a summarization is this when we order the stock price by day, we get what is called a time series. This will be a central data structure. In this course, time series methods often assume that your data is sampled at equally spaced intervals. So by having daily data, we automatically meet this constraint. Note that there is still one column of our data that we haven’t yet discussed, which is the adjusted close, we’ll discuss this in a later lecture since there are some important points to mention. But for now, let’s focus on how to actually look at this data in Python. Thanks for listening and I’ll see you in the next lecture.

5. Understanding Financial Data (Code)

In this lecture, we are going to look at a notebook for loading in and understanding financial data. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So first, we’re going to start by downloading our data set. Note that this is not the data set we just created, but rather a data set of S&P stocks from Kaggle. This is basically the same as our data set, except I wanted to use this one since it has some nice properties, such as missing data. In addition, it doesn’t have the adjust the clothes column, so there is one less thing to worry about. By the way, this Kagle repository is the same place that I got that list of S&P symbols from. So if you want to check out the Kagle page to learn more about the data, please do so. So the next thing we’re going to do is import pandas. Next, we load in the data set using pedigreed CSFI. Next, we do a DFG head to look at the first few rows of the data frame, as you can see, it has the same columns as the data we just created, except for the adjusted close column. So let’s remind ourselves about what we are seeing here. We have the open, high, low and closed columns which refer to certain prices for the given stocks on the specified day. Open is the price at the beginning of the day, closes the price at the end of the day, high is the maximum price for that day and low is the minimum price for that day. Volume is the number of shares of this stock traded on that day. Next, we do ADR info so we can check the tape of each column as well as the type on the index when you’re working with certain kinds of time series, sometimes this is information that you care about, as you will see later on in this course. Next, just as a basic sanity check, we can grab all the unique names from the name column. So you might want to look at this list and make sure that your favorite S&P stocks are included. Next, we can check the shape of the preceding array to ensure that it’s approximately 500 elements in length. As you can see, it has 505 five elements. Note that the S&P 500 does not actually have exactly 500 stocks. 500 is just an approximate number. Next, let’s say we want to grab all the stock prices for a specific company, for this example, let’s say we want IBM, how can we do that? Well, we just filter on all the rows where the name column is equal to IBM. As you can see, the result contains only rows where the name is IBM. Next, let’s assign the sub data frame containing only the name IBM to a variable called IBM. If we want to do something like grab all the clothes prices, we can simply use the square bracket notation and grab the clothes column. This returns a series of closed prices. Not that we can call the plot function a directly on a series object, so there’s no need to import matplotlib, although this calls matplotlib under the hood. As you can see, IBM stock price drops quite a bit during this time period. Now, you might ask, what is this time period? Well, we can check if you didn’t look at the documentation earlier, we can find this information ourselves by checking the date column. We can call the main function and the max function. And this will return the minimum value, which is February eight, 2013, and the maximum value, which is February seven. And. One thing to consider is that the current format of our data frame is not all that useful. Each stock is separate, but we know that there’s a relationship between different rows of our data frame, and that is their dates. For example, we have the price for Google and the price for IBM on the same days. But these days are completely different rows in the data frame for different stocks in finance. It’s customary to work with the closed price by default. So what we would like to have is a single data free organized by date, where each column is the close price for a different stock. As an example, let’s consider this data frame, as you can see, the data is used as the ROE index of the data for each column is named after a specific stocktake and each cell in the data frame is the close price on a specific date for a specific stock. The question is, can we convert our existing data frame into a data frame that has this different format? The answer is yes. So in the next few blocks of code, I’m going to show you how to construct such a data frame. First, we’re going to start with a completely empty data frame with no columns whatsoever, just an index as mentioned. We want to use the date as the index to get this started. I’m going to call derange passing in the min and max dates that I found earlier. This will create a date time index object. One important thing I want you to notice about this object is that it has some information other than just the dates themselves. There is an attribute called Freak Show It for Frequency, which tells us that each value in the index is a different day. These stands for Daily. Next, I’m going to create a data frame called Close Prices by calling the constructor PD data free. The only argument I’m going to pass into the constructor is the index argument for which I’m going to pass in the daytime index I just created. So if I call closed prices dot head, you’ll see that no columns are printed out, just the index. This data frame currently contains no columns. Of course, the next step is going to be to fill in these columns with the data from our original data frame. There is a slight problem with this, which is that in our original data frame, the dates are just a regular column. They are not part of the index. This actually makes sense for that data frame, since the dates are not unique. Multiple rows have the same date simply because they contain prices for different stocks. We know that we’re going to want to do something like a database joint where we join at two tables on some index, specifically the date. This is just like how you would do a joint in school. In fact, PANDAS has an analogous function called Join, which does exactly the same thing. In order for this to work, what we would like to do is create a temporary data frame for each stock, one at a time. This temporary data frame should contain only a single column, the close price for a specific stock, and as its index it should have the corresponding dates. So the next line demonstrates how to do that. Specifically, we’re going to call the data frame constructor as arguments we’re going to pass in data equal to the close price from the IBM substate of frame for the index argument will pass in EBM square bracket date. For the columns, argument will pass in the string EBM, since we want the column name for this column to be I.B.M., we don’t want the column name to be something generic like clothes, since all of the columns will be closed prices. Next, we call the head function on day two to ensure that this data frame has the format we want and it looks like it does. The data is being used as the index and we have a single column containing close prices. And the name of that column is IBM. OK, so the next thing we want to do is to do this for all of the symbols in our original data frame. So next we’re going to loop through every symbol and our symbols list. Inside the loop, we grab the sub data frame containing only the symbol in the name column. Next, we create a new temporary data frame called D.F. Temp using the same code we discussed earlier. Finally, we call the joint function on the closed prices data frame passing in the F temp. We assign this to close prices. So on every round of the loop, closed prices will accumulate each new symbol. Next, we call the head function to ensure that our data frame looks as it should, we have the date as the index in each column is a different stocktake. And of course, the cells are the closed prices. Note how there are a lot of fans recall that when we created our date index, we specified a date range. This includes all days from the start date to the end date. But of course, not all of these days, our trading days, you may know from experience that markets are closed on weekends and holidays. There are usually around two hundred fifty two trading days per year. We’ll see later how we can remove these nations from the data frame. Next, let’s check the info function again to see a summary of our data, as you can see, it’s a data frame. It has a date time index with one thousand eight hundred twenty six entries with a frequency ID, which means daily. It has 505 columns from A-L to Zietz. The type is Flow 64, which makes sense. The next step is to save our data frame. We call the 2C ASV function and we save this file to ESP five hundred underscore close. That’s ESV. Next, we use the Lennox Head Command to see what’s in our file. Notice how whenever there were nans, these just show up as nothing in the CSFI. Next, let’s see how we would load in RSV from a file into a data frame. This is not as straightforward as just calling Pedigree’s ASV, since there are a few arguments we want to pay attention to. Let’s call this data frame close to. So for this data frame, we want to make sure that the first column in RSV gets assigned the index. Otherwise what would happen is the index would just be an integer like 012 and so forth. So the date would become a regular data column. We don’t want that. So we specify index call equals zero. We also want these dates to be converted to dates and not to be treated like strings, so we pass in the argument past dates equals true. If we do close to [REMOVED], we can see that our data frame has been loaded and successfully. Next, we can do a close to dot info to get a summary of our loaded in data frame. Notice how is the same as before, except that it doesn’t know that the frequency of the dates are daily. This is not a huge problem unless for some reason you need this information in a downstream computation.

6. Dealing with Missing Data

In this lecture, we are going to discuss how to deal with missing data. The answer is not what you may intuitively assume. But first, let’s talk about the fundamental question. Why does data go missing in the first place? There are several reasons why this might happen. First, let’s consider a stock price like this where your data just abruptly stops and then never appears again. How might this happen? Well, this might happen if a company gets bought by another company. An example of this is LinkedIn, which used to trade under the Ticker LNKD. As you recall, LinkedIn was acquired by Microsoft in December 2016. The LNKD stock ticker was delisted from the New York Stock Exchange. How about something like this, in this case, the stock price has no data before a certain date, only after what might cause a situation like this? Well, this can happen if the company did not exist before this date, but we would like to include it in our analysis. And it just so happens that no data exists because no shares existed. How about something like this, in this situation, it seems that data is missing in between. So the company didn’t get acquired, but also the company existed before the missing data. This can happen with small companies with low liquidity when no shares are being traded, when no shares are being traded. There is no price. Another situation that can happen is that your data is simply bad, maybe the API you were using had some kind of technical glitch and they couldn’t store some data that they should have. This kind of thing happens very frequently in production systems, as you know, if you have ever worked as a software engineer. So what can you do about missing data? Well, let’s talk about the solution you probably are thinking of already. If you’re not thinking, well, let this be an instruction to begin now. So take a moment and think about what would be your solution to this problem of missing data. If you’re like me, then the solution you probably thought of was to use linear interpolation, that is, if you have two data points with missing data in between, simply draw a straight line to connect them together. You’ll notice that I’ve been setting you up to explain why this is not a good solution. I mean, this seems like a pretty mundane a standard thing to do in data analysis. So why would we not want to do this? Here are some reasons. First, let’s suppose we are at some point, just after the missing data was filled in, let’s say, right here, what’s the problem? The problem is that I’ve used future data to fill in past data. This is a big no no in finance. Using future data in your algorithms will pollute your results. Finance is a field in which predicting the future is very difficult. So a signal like this is extremely informative. Just to be clear, since it might not be obvious at first, why is this using data from the future? Well, consider the slope of this line. The slope of the line is the second point minus the first point divided by the difference in time. But how do we know the second point? If we are at any time before the second point, it is not allowable to use the second point because that second point is in the future. All right, so linear interpolation is bad. What do we do instead? In fact, the best solution is to simply copy the previously known value forward in time. We call this forward filling. This might seem strange, but it will be justified in a future lecture when we discuss the random walk hypothesis. But what happens if there is missing data in the front where there does not exist any past data to copy from? In this case, the best we can do is called backwood filling. In fact, these operations are so common that Panis has functions that do exactly these things forward filling and backward filling. Automatically, we use the fill in a function, passing in the arguments, F fill and B fill. Note that we only backward fill after we forward fill, which is why you see big jumps only after the long horizontal lines. So just to recap, our first choice is to forward fill, and failing that, our second choice is to backward, to fill. So if you imagine if you have a data frame like this, which starts out as in a three, five, two and A and A and one first we forward fill. So that becomes in a three, five to two to one and then we backward fill. So it becomes three three five, two, two, two, one. As a final note, I want to mention that days like weekends and holidays are not considered missing data for these days. If they happen to appear in our data for some reason, we can simply drop the entire row. For these days, all rose will be missing. And so, for example, the time difference between a Friday and Monday will be considered as one day, even though time wise it’s not actually one day. In other words, when we look at a time series of stock prices, the data points are not actually equally spaced in real time. They’re equally spaced in trading time.

7. Dealing with Missing Data (Code)

In this lecture, we are going to look at some code for how to deal with missing data, as well as continue looking at some other basics of dealing with financial data. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Note that this is a continuation of the same notebook we’d been looking at previously. So first, we’re going to start by using our clothes Price’s data set and calling the plot function directly on the series given by the IBM column. Note that there is some missing data here, as you can see from the gaps in the line chart. However, recall that this may be because of how we created our data set. We created the data set by creating a date range, which included all the days between a specified start date and end date. Of course, this necessarily includes weekends and holidays. The next thing I want to show you how to do is how to get rid of all the rows in which the entire row is missing. These are days which likely correspond to non-trading days. And even if they are not non-trading days, we can’t do anything with these days anyway. As you may know, this can be accomplished by using the drop and a function. But if you just call drop in a without any arguments, it will not do what you want. Drop in a will drop any row with any missing data by default. We only want to drop the rows where all the data is missing. So first we specify access equals zero, which means drop rows and not columns. We say how equals all so that only rows in which all data is missing are dropped. Finally, we say in place equals true so that we do these operations on the existing data frame. Next, we plot the IBM close prices again, and at least the visible gaps which were pretty evident, are now gone. We can check how many missing values are still in the data set by using the IS and function, this returns a bullion in every location. True, if the cell is in a and false, otherwise, recall that in Python a true equals one and false equals zero. So if we simply sum all the values we should get how many missing values there are. If we call some once, you will notice that it only comes along one axis at a time. So now we have the number of missing values for each individual stock. Since there are too many stocks to look at all at once, the next thing we can do is simply some the result. So in effect, we call some twice and that tells us how many missing values there are in total. As you can see, there are about seventeen thousand missing values. Now, we know that any stock for which there is no initial value must be backwards filled. So let’s try to figure out how many stocks fit that scenario. We can do so by checking how many stocks have a missing value in the first row of the data frame that can be accomplished by using the ILO function. We call Ilocos Zero to get the first row of the data frame. We call is N.A to get a Boolean for whether the data is missing or not. And finally, we call some to get the total number of missing values in this row. As you can see, there are twenty nine stocks for which there is no initial value and these are the ones which must be backward filled. As mentioned before, we do any backwards filling, we must forward fill, we can do that by calling the fill in a function saying method equals f fill. Durex pandas to use forward filling and in place equals true means the operation is completed on our existing data frame. After we do this, we should check again to see how many missing values there are now. Interestingly, we see that a majority of the missing values from before are still missing. This indicates that many of the companies in our list are for some reason, are missing a lot of data from the start of this data said. But recall that this data set is from Kagle, this may not be the case for the data set we collected ourselves from Yahoo! Finance. I’ve noticed that this was at least the case for Google. It’s probably the case for quite a few other stocks as well, since Google has certainly existed long before the start date of this data set. So how do we backfill? Well, we call the exact same function fill in a from before, except now we pass in method equals Beeville. After this, when we check the number of missing values, we see that it is now zero as expected. The next thing I’d like to do is plot all of our stock prices on the same chart for this plot. I’m going to say legend equals false, since showing 500 items in the legend is a bit too much. I’m also going to set fixes equals 10 10 so that we can see the plot more easily. So what do we see? Well, they look like a bunch of stocks, which is nice. There is at least one stock where we can see this backwards Phil behavior. So it’s this great one over here. As you can see, there is over a year’s worth of missing data for this stock. One common operation and finance is to plot the relative stock price over time. This allows us to more easily compare growth or lack thereof of different stocks. We can accomplish this by dividing each column of stocks by its initial value. Of course, the initial value divided by the initial value will become one, and then each subsequent value will be the ratio between the current value and the initial value. This allows us to easily see the cumulative return on each stock, which we will define more rigorously in another lecture. For example, when we see a stock on this normalized chart that goes up to 20, that means if we invested in that stock, our investment would have gone up by a factor of 20. So we can accomplish this normalization simply by dividing the data frame by the first row of the data frame. That’s close price I lokke zero. Note that broadcasting happens automatically so there’s no need to do a for loop. Finally, we plot our normalized prices and in fact, we do see a stock that goes up to about 20 as an exercise, try to figure out which company this corresponds to.

8. Returns

In this lecture, we are going to talk about returns. People often talk about the return on investment. That’s exactly the kind of return we want to discuss. What does return mean? How do we calculate it? Are there different kinds of returns? We will answer these questions in this lecture. Returns can generally be thought of as a percentage. I like to think of them as the same way I think of a sale when you’re shopping, but with the signs reversed. So, for example, if you see a product that costs 100 dollars normally but is 20 percent off, that means you’ll pay eighty dollars. Similarly, if you invest 100 dollars today and got back on the 80 dollars tomorrow, then your return would be minus 20 percent. Of course, that’s not a very good return since we would usually like our investment to increase in value. To be more accurate, we defined the net return as follows returns are always defined over some time period, whether that’s over a day, a month, a year or even a minute. So given some time period, assuming no dividends, the net return is calculated as the final price, minus the initial price all divided by the initial price using algebra. It’s easy to see that this is equivalent to the final price divided by the initial price minus one. And remember, this is called the net return, but sometimes we simply refer to it as the return. So if I ever talk about the return without qualifying, what kind of return I’m talking about, you can assume, is this. Note that another way of writing the return, which I find to be more expressive, is to index the prices by time. So the net return at time t is equal to at time t divided by time T minus one. This is also equivalent to a time T minus at time T minus one all over time. T minus one. Now, even though this is pretty trivial, let’s do a simple example just to make sure everyone is on the same page. So suppose that my initial investment is one hundred dollars. I get back one hundred twenty dollars. Therefore, my net return is one hundred twenty minus one hundred, all over one hundred. That’s equal to twenty over one hundred, which is equal to 20 percent. It’s clear that if we lose money, our net return will be negative. So, for example, if my initial investment is 100 dollars and I only get back eighty dollars, then my net return is eighty minus one hundred, all divided by one hundred, which is equal to minus 20 percent. Sometimes we like to express the return as a ratio of the new value to the old value. This is called the simple gross return. We don’t normally have a symbol for this, but rather we just write it as one plus R t, where R of T is the net return. This is useful because it tells us relatively how much our investment has increased or decreased compared to the number one. For example, if my initial investment is one hundred dollars and I get back one hundred twenty dollars, then my gross return is one point two. That is, I got back one point two times what I originally put in. If my initial investment is one hundred dollars and I get back eighty dollars, then Mike Gross return is zero point eight. That is, I got back zero point eight times what I originally put in. And of course if my investment doesn’t change, then my gross return is just one. It should be easy to see that this is exactly related to the net return, which we just discussed and all I’ve done is added one to both sides. Finally, we have the log return, also known as the continuously compounded return, the log return is very important, yet very strange. We don’t walk into a store and say, what can I buy for eight point five log dollars? We’ll see later in this course why log returns and log prices are convenient for modeling the movement of stock prices. Until then, let’s try to understand the definition. The log return is just the log of the gross return. In other words, it’s log of one plus afte, which is the same as log of T divided by P of T minus one. And that’s also the same as log of T minus log of time T minus one. Note that it’s sometimes conventional to use lowercase letters for log returns and log prices. So little r t is equal to log of one plus big RV and little Patty is equal to log of Big Patty. And so the log return is just the difference in log prices. Intuitively, we can see why log returns and log prices are convenient instead of having to do multiplication and division. We can simply do addition and subtraction. So the log is an interesting function, especially in the context of finance, as you’ll soon see, returns generally take on very small values, very close to zero. In fact, financial engineers often don’t even discuss returns in natural units like percentages because they’re often in the realm of fractions of percentages. In other words, if you use regular numbers, you’d see lots of zeros after the decimal point. For this reason, financial engineers often use units called basis points, where one basis point is one one hundredth of one percent. In other words, a basis point is one percent of one percent. And so returns taking on such small values has an interesting effect on log returns. Specifically, we know that for small X, X is approximately equal to log of one plus X. If you’re not convinced of this, you’re encouraged to draw a graph such as this one as an exercise. This means that whether you are looking at log returns or non logs returns, they will all be approximately equal. That means when we look at their distributions, they will also be distributed roughly in the same way. Let’s do an example with cumulative returns. Suppose I’m given the daily return over an entire year. That’s two hundred fifty two returns and an initial price p0. I want to calculate P of 252. That is the price at the end of the year. It’s important to remember that returns compound. In fact, we can understand this simply by rearranging the equation for the return we had earlier. So P one is equal to zero times one plus one, P two is equal to one times one plus our two, but that’s also equal to zero times one plus our one times one plus our two. And therefore 252 is equal to zero times one plus our one times one plus our two all the way up to times one plus are 250 to. Suppose we wanted to know the average daily return. That is what return, if I got the same return daily over the year, would result in the same final value P. of 252. Well, we can calculate that by saying P 252 is equal to zero times one plus R to the power to 52. This is the same formula from earlier, except we can combine all that one plus hours together since now we want the same R. We can also equate this to our previous expression so that we can find out more in terms of the daily returns over the year after rearranging we get that are is equal to the product for Michael’s one to 252 of one, plus our eye, all to the power one divided by 250 to all minus one. As you may know, we call one plus R, the geometric mean of all the one plus our eyes. Now, let’s consider what would happen if we use the log returns instead, if we take the log of both sides of our previous expression, we get that two hundred fifty two times log of one plus R is equal to the sum of all the one plus our eyes. Therefore, a little R is just the arithmetic mean of all the daily log returns. As a final note for this lecture, I want to say that depending on where you look, you may see different definitions for different types of returns. Generally speaking, I want to avoid a terminology in this cause as much as possible and pay most of our attention to the mathematics, since that’s the most important and unambiguous. So if you’ve seen other definitions in the past, just put those aside for this chorus.

9. Adjusted Close, Stock Splits, and Dividends

At this point, we are going to take a slight detour before we look at how to calculate returns and code. So if you recall, there is one column in our data set which we haven’t really discussed, and that’s the adjusted close column. Clearly, this column is related to the close column somehow, and it does some kind of adjustment. But what is this adjustment? Should we care? Which one should we use? These questions will be answered in this lecture. So the standard explanation of the just a close call is that it takes the close column and adjusts it for stock splits and dividend payments. Most people will just leave it at that and move on. For us, there are some peculiarities that I think are interesting to pay attention to. Note that this discussion is optional. So if you don’t understand it 100 percent, know that it’s not necessary to move on with the rest of the course. First, let’s consider stock splits. What is a stock split to motivate this discussion? Let’s suppose you want to buy some shares of a stock. It would probably be more helpful if you’ve had experience buying stocks in the past. If you haven’t, then maybe now would be a good time to give that a try. So when you are buying a share, you have to buy them and sell them in whole units. You can’t do something like buy one quarter of a share of Apple stock. So what’s the problem with buying whole units? Well, what if the share price goes so high, say one hundred thousand dollars that it’s infeasible for most people to buy any shares? Well, that’s a problem we would say that this stock is less liquid, it makes trading the stock difficult for many investors, whether you own that stock you want to sell or you are interested in buying it. What a company can do in this scenario is a stock split. As an example, suppose I do a two for one split, that means for every share that currently exists, I will create two shares in order for the total value to be equivalent. I must also have the price. So what might that look like? Let’s say I own one hundred shares of a stock that’s worth one hundred dollars per share. Now I do a two for one split after the split. I will own two hundred shares, but each share will be worth fifty dollars. In both cases I own it. Ten thousand dollars worth of this stock. Only the number of shares and the share price have changed. The total amount that I own has not. So what implications does this have on the stock price? Well, remember that we usually like to think of the close price as a time series. What happens during the stock split? Well, as we discussed, the stock price will go down by a fraction. So if we do a two for one split, the stock price will be divided by two. If we do a 10 for one split, the stock price will be divided by 10. Two famous examples of stock splits are Google and Apple, which both happened, incidentally, in the same year 2014. Obviously, this presents a problem if we use closed prices to calculate the return, if you naively use a formula like price at time, T minus price at time, C minus one all over priced at times you minus one, you will get the wrong answer. If the company did a two for one stock split on that day, then it will appear that the stock went down by 50 percent, which would obviously be very bad. But that’s clearly not what has happened in reality. Therefore, if we are to calculate actual returns, we have to adjust the close prices so that they are relative to the price either before or after the stock split. In practice, when you download a stock price data set, the close prices are adjusted such that the closed price and the adjusted closed price are the same in the final row of the data set. After that, everything is adjusted, going backwards. So here is an example of some synthetic data I generated that shows us visually what happens when stock splits occur. As you can see, the value of closed and adjusted close at the final timestep are the same. They are also the same going backwards up to the point of the latest stock split. At that point, there was a two for one stock split. Beyond that point, the true closed price is double that of the adjusted closed price. But notice how if you use the adjusted closed price to calculate the return, you will get the true return. Note that this graph has two stock splits, so if we keep going backwards, we see that the next latest stock split was a three for one split. At this point, you should ask yourself, what number do we need to divide by in order to get the adjusted close price? I’ll give you a moment to think about this. So if you like, please pause the video until you think you have the answer. Remember that for the latest stock split, it was a two for one split and we divide it by two. Now we have a three for one split. So what do we divide by to get the adjusted close price beyond this split? In fact, the answer is six. That’s because we are going backwards and we have to account for not just this stock split, but all future stock splits in this data set. So beyond the second stock split, the factor between the true closed price and the adjusted closed price is six. Now, stock splits are not the only factor that affects the adjusted close price, the other factor that affects the adjusted close price is dividend payments. Dividend payments are money that you get paid in cash simply for owning a share of the stock. As an example of this, we can look up Apple’s dividend payments. These are, in fact, posted on Apple’s own website. So feel free to check out investor dot apple dot com slash dividend history for the most recent dividend payment amounts. What’s quite interesting, which nicely ties into our topic of discussion in our examples with Apple and Google, is that Google famously does not pay dividends. The reason for this is Google believes they can provide more value to investors by using that money to invest in themselves. That is, by using that money to do more research, support their infrastructure and so forth. In this course, we will pretty much ignore dividends, so if you want to research this on your own, you can feel free to do that. All of the techniques in this course still apply. Dividends generally just make our computations more complicated so they will be considered outside the scope of this course. But to give you a basic example, suppose that a stock closes at 20 dollars on some particular day. After closing, the company announces that they will pay a one dollar dividend per share. The adjusted closing price will then be twenty dollars minus one dollar, which is equal to nineteen dollars. To give you some idea of why calculations become annoying when we consider dividends, we can look at how one might calculate returns when dividends are present. The gross return, which, if you recall, appears in one plus form, is defined as follows. It’s one plus R of T equal to T plus D of T, all divided by P of T minus one. And this is where DFT is the dividend payment at time T, therefore the net return is just the right hand side, minus one. Note that some resources will suggest calculating the return from the adjusted close price, which is not algebraically equivalent. To give you an example of this, consider a few rows from the stock ticker, S&P. The last dividend was paid on May 21 at two thousand nineteen. So every closed price after that date is equal to the adjusted clothes on May 20. We see that the adjusted clothes is different from the clothes price. In fact, two point eight three minus zero point one five is two point sixty eight. So the adjusted clothes was calculated by subtracting the dividend from the close price. This is the same as I showed you earlier. So if we calculate a return using the method that you will see and finance textbooks, you will get two point four nine plus zero point one five, all divided by two point eight three, all minus one, which is equal to minus zero point zero six seven. But if you use the adjusted clothes, you will get two point four nine divided by two point six eight minus one, which is equal to minus zero point zero seven one. In other words, depending on which method you use, you will get a different answer. So why did I want to mention stock splits and dividends? Well, there is a slight nuance when we consider the data sets that will be using. It turns out that some APIs, such as the Yahoo! Finance API and for the data sets that we have chosen for this course, the closed price has already been adjusted for stock splits. In fact, I’ll demonstrate this in the next lecture. Therefore, the only difference between the adjusted, closed and closed price in our data sets is dividend payments. So although I just gave you the official explanation for what the adjusted close column should be, this is not what we will see in our data. Our data will reflect dividend payments only anyway. I understand if this is a slightly boring or confusing topic, so I would consider this discussion to be optional. And basically the only thing you have to keep in mind is that we are going to use the closed price rather than the adjusted closed price. We will not account for dividend payments since that would simply require additional housekeeping. However, in your real work, you would want to be as accurate as possible. Ultimately, in any discussion about whether you should use the closed price or the adjusted closed price in your back testing strategy, what you want to consider is how to make your simulation as close to the real world as possible. Keep in mind that the close price is the actual price and dividend payments are real money. They get paid into your account. Personally, I’ve seen both. I’ve seen people say you should use the closed price and I’ve seen people say you should use the adjusted closed price. In both cases, however, we do want to adjust the close price for stock splits. Luckily, our data already accounts for this. The dilemma is based around dividends only. We will be using the non adjusted close price, which means not adjusted for dividends. However, note that everything in this course can be done without any loss of generality. Whether you want to use adjusted clothes or just the regular clothes, it’s up to you. The code itself will not change. While you would have to do is change the caller name. One reason in our class to use the regular clothes price is because one of our data sets does not have any adjusted close column. Therefore, using the regular clothes column keeps things simpler.

3. Time Series Analysis

1. Time Series Analysis Section Introduction

In this lecture, we are going to be introduced to the next section of this course, which is all about time, series analysis, fun fact, this is yet another section of this course that became way more massive than I had anticipated. So in the previous section, our main goal was more exploratory data analysis and static analysis. We wanted to understand stock prices and stock returns, assuming that their distributions stayed constant over time. In this section of the course, we’re going to turn to the question of how do we model how the stock price evolves over time? This section will focus on classical statistical techniques in Time series analysis. We’ll be focusing on two main areas, the whole winter’s exponential smoothing model and Arima. Both of these are staples in financial analysis and financial engineering. So I’m sure you’re not surprised to see them. If you’ve taken my machine learning courses in the past, then you’ll recognize exponential smoothing as something we’ve seen quite a few times, it appears in the Adam Optimizer, in back to normalization and in reinforcement learning. However, you’ll see how hole in winters extend this basic idea to build a pretty powerful forecasting model for Time series analysis. Arima, on the other hand, is a full fledged machine learning model. The basis for Arima is linear regression. So if you want to get the most out of the section, you want to make sure you at least have a very good intuition for how linear regression works. One of the interesting aspects of Time series forecasting with stock prices is that it is deceptively difficult. This will be one of the main themes of this section. You’ve heard me repeat the phrase, all data is the same in the past. This is one of the most important rules that I repeat to my students, because it’s one of the concepts that a lot of people who are new to machine learning get stuck with as an example of this. Suppose I use some regression algorithm to predict a person’s height from their weight. Let’s just say it’s a neural network regression. Now, let’s say I would like to use my neural network regression algorithm on some other kind of data set. Let’s say I want to predict your exam grade based on how many hours you studied. I think both you and I would agree that the same algorithm could be used in both cases. I can just take the neural network regression class from Saikat, learn and use the same code for both data sets. Why? Because all data is the same. Without this rule, library such as cyclosarin learn would not exist. So then the question becomes, can we do the same thing with Time series? We know that Arima models are pretty good at times, Houris models, they do sales forecasts, weather and all kinds of time series data. If it’s true that all data is the same, then we should be able to plug in stock prices and see similarly good results. The only problem with this idea is that it’s not exactly true. One important thing you will learn in this section is that there is something fundamental about stock price data that makes it unlike other kinds of time series data. Some people even think stock prices are impossible to predict. So that’s going to be an important question to tackle in this section. What is it about the nature of stock prices that makes them so difficult? Now, you might wonder, does this break my rule, all data is the same if we can predict sales data with a rhema, but the predictions for stock prices don’t quite look the same. Does this mean that all data is the same as false? The answer is no. All data is the same does not say anything about the accuracy of the result in computer vision. For example, we use CNN to classify images. The same kind of concerns apply to Tumnus as they do to images of x rays, for example. But if you know anything about machine learning, then you know that amnesty is a much easier problem than classifying x rays. All data is the same is more about the code, not about the nature of the data. So you have to be able to separate these two things in your mind. On one hand, you have the code. When you’re using Arima, you’re using a Arima. There isn’t one kind of Arima for sales data and a different kind of a for stock prices. It’s all the same Arima. On the other hand, you have domain knowledge that is understanding your specific field or your specific industry, if you work in, say, fashion, then you’ll want to know specific knowledge about the fashion industry and how the different seasons affect sales and product development and so forth. If you work in finance, you don’t know anything about fashion because that’s not your job. Instead, in finance, you study finance. Your job is to know about money and how it flows through various actors in a financial system. So that’s important separating the algorithm from the domain knowledge. These are two different things. The algorithm is the same no matter the domain, the domain knowledge helps you attribute meaning to the results of those algorithms. One takeaway theme from this section is that it’s all about modeling, machine learning is often focus on making accurate predictions no matter the model. It doesn’t matter whether the model is explainable or intuitive as long as it works. Statistics is all about having an explainable and intuitive model, and it’s willing to sacrifice predictive accuracy to accomplish this. So long story short, are you going to use the section to precisely forecast the stock returns for every stock over the next month so that you can pick the one with the highest return and make lots of money? The answer is no. If you could perfectly predict stock returns, there would be no need for concepts such as portfolios, diversification and so forth, but it’s important to remember that this is not the purpose, the purpose of this section. It goes much deeper than that. Unlike a machine learning course which is focused on the algorithms, this course is actually focused on domain knowledge. Your job will be to use the models in this section to gain a better understanding of the data so that you understand why this isn’t the right question to ask.

10. Simple Exponential Smoothing for Forecasting (Code)

In this lecture, we are going to look at simple exponential smoothing in code, as you recall, this is the same operation as the exponentially weighted moving average. The main difference is philosophical. We are now treating this as a forecasting model rather than just a method of calculating a moving average. In addition, instead of using pandas, we’ll be using the stats models library. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, sort of start. We’re going to import the class simple exp smoothing from the whole winters’ module in stat’s models. Next, we’re going to instantiate an instance of our new class passing in our passengers data into the constructor, we’ll call this model Se’s. Notice that we get a warning which says no frequency information was provided, so in a word frequency, EMS will be used. The reason for this is when we load it in our data frame. Panas didn’t automatically assign a frequency to the index. In the next block, I’m going to print out the index of our data frame. And we can see that the frequency is known. Luckily, we can fix this pretty easily. We can just assign it DFT Index d’Afrique to the string Ms. MS means months. And if you want to know how I figured this out, you can check the link I’ve left above, which lists out all the different possible frequency strings that you can assign to date range indices. Next, I’m going to create another instance of our simple smoothing class with our newly updated data frame. As you can see, we no longer get the above warning. The next step is to call the fit function on our model, as I mentioned in the theory lecture, we are going to use a fixed alpha and we’re going to set optimized equal to false. This is just so that it does the same calculation we did previously, which allows us to compare the outputs. So let’s assign the return value to Reg’s. Next, I’m going to print out Rez so we know what we are working with. As you can see, it’s a whole winter’s results rapper, so if you want to look up the documentation for this class now, you can. The next step is to call the predictive function on a results object, since I want the predictions for our entire data set. I’m going to pass in a start date equal to the first row index of our data frame and end date, equal to the last row index of our data frame. As you can see, this returns upand a series. In the next block, I’m going to call the same function again, but I’m going to assign it to a new column in our data frame called Se’s. Next, I want to demonstrate that the predicts function, since we called it for only in sample dates, actually returns the same thing as the Fidrych values attribute. So in the next block, I use the function and I’ll close passing in these two strings since the result is true. That means the predicate function returns the same values as the fitted values attribute. Next, I’m going to plot our data frame. Note that this is the entire data frame, including everything we put in there previously. Now, right away, you should notice something strange, the results from our CBS model are different from pandas. They seem to be shifted up by one. Why is this? Let’s do a dot head to get an idea of what’s happening. As you can see for both women and says the first value is 112, which is just the first value of our Time series for either WMA, the next value is one thirteen point two. But for Se’s, the value 112 gets repeated again. After that, we have won thirteen point two. So what’s going on here? Let’s try to shift the sex column back by one to check if the rest of the values line up. In the next block, we’re going to plot WEMA against one. As you can see, the values do indeed line up as suspected, however, you’ll notice that I have this comment in here in which I tell you very loudly, no, do not do this. In fact, this is wrong. Now, I won’t name any names, but I’ve seen a few other people who do this who happen to be very popular among beginners. So why is this wrong? Recall that the forecasting model is defined as slightly differently from the traditional IWM. As you recall from the theory lecture, the forecasting time index is actually moved up by one step. The IWM is represented by the level, but the prediction we have is actually assigns the level at the previous time step. In other words, the SES model should be lagging behind by one time step. So what we had plotted originally was correct. You’ll see in our later examples why this is true. Remember that if we shift this, then we have to shift everything, but you’ll see that with hot winters, the foam model, there is no need for shifting because it’s actually the correct model and it will match up very nicely. If we shift this, then we have to shift that to be consistent. And you’ll see that would be completely wrong. All right, so the next thing we’re going to do in this lecture is to treat this more like a machine learning problem, that is we’re going to split up our data, set into train and test, and we’re actually going to do a forecast. So I’ve sat and test 12 and I’ve set the train set to be everything up to just before the last 12 data points. I have therefore set the test set to be the last 12 data points. Next, I’m going to recreate the simple XP smoothing object, but this time with only the train set. Again, I’m going to give this the variable name X next I call the fit function. Notice how this time I’m not going to pass in alpha or set optimized equal to false. What this function will now do is find the best alpha so that it minimizes the squared error over the Transat. Next, I’m going to assign the predictions from my model back to our original data frame D.F. To do this, I’m going to set resident fitted values to go from zero up to minus and test. This will be assigned to the column sees fit it. Next, I’m going to call resident forecasts for an end test time steps. I’m going to assign this to the data frame under the same column in the final end test rose. Next, I’m going to plot the passengers and seats fitted columns. As you can see, the forecast is indeed a horizontal straight line, as promised. Note that again for the train set. The prediction looks like it’s lagging behind by one step. You, again, may be tempted to shift this back so that they line up perfectly. But remember, this is not the correct thing to do. A simple reason to remember why that is, is because you know that the model stores the dates, so we have to presume that the model lines up those dates correctly. Now, if you did shift it back philosophically, think about what that means. That means this simple exponential smoothing model makes it nearly perfect predictions, which means that there is no further work to be done. We don’t have to model the trend that we don’t have to model the seasonal component. And of course, that would not make any sense. Furthermore, looking at this plot, we can try to guess what the value of Alpha might be when our Alpha was zero point to the one we set manually, the IWM looked very smooth and didn’t track the original signal that fast. On the other hand, for this fitted alpha, we can see that the model is reacting very quickly to the original Time series, which would indicate that it cares more about the most recent sample and less about the previous average. Therefore, we might guess that this alpha is very close to one rather than being closer to zero. In the next block of code, we check what the value of Alpha is by printing resident operands. As you can see, the result is, in fact, exactly what that means are fit and model is simply doing the naïf forecast, i.e. it simply copies the last known value in the series.

11. Holt’s Linear Trend Model (Theory)

In this lecture, we are going to discuss holds linear trend model. Previously we looked at simple exponential smoothing and we noted that our forecasts were always a straight horizontal line, holds linear trend model, extends the exponential smoothing model so that we can capture and forecast trends. Let’s start by considering what linear trend might look like. Well, linear means line. We know that the equation for a line in general is Y equals M, X plus B, let’s suppose that instead we say Y, Sebti equals slope times T plus. Why not? This doesn’t change anything. All we’ve done is replaced X with T and be with Y not. As you know, the coefficient in front of T is called the slope and the constant term by itself is called the intercept. Clearly it is the value of Y at time T equals zero, hence the symbol y not or Y zero. Note that the slope is the amount that y changes when t increases by one. Understanding this kind of equation is a crucial step in understanding Holtze linear trend model. The best way to understand holds linear trend model, in my opinion, is to simply look at the model equations and to try and decipher each component one at a time. The first thing you’ll notice is that it’s very similar to the simple exponential smoothing equations in component form. That’s why it was so important to rearrange them in that way. You’ll notice that, whereas the simple exponential smoothing model had two equations, we now have three equations. So we’ve added one more equation, specifically one more equation for modeling the trend. So now we have three equations in total, one for the forecast, which we already had, one for the level which we already had, and one new equation for the trend. Let’s start by studying the forecast equation, as you can see, this is nothing but the equation for a line. Of course, we now have different symbols, but I know you’re not going to let that intimidate you as before. We have the level which represents some sort of average value. But now we also have a term that increases linearly with H, which is the number of steps in the forecast. Clearly, this means that B of T is the trend, or in other words, the slope that is elevated, the level is the starting value of the forecast and then the forecast increases by the amount B of T for each step into the future that we forecast, of course, but can also be negative. So if you increase by a negative amount, then you’re actually decreasing. The key point is, thanks to this new linear trend term, our forecast is no longer a horizontal line, but a line in any direction. This justifies the name holds linear trend model. Let’s now study the level equation again, this is very similar to the level equation from before. The only difference is on the far right, which represents this moving average value. Remember that the prediction is now made up of two components, the level and the trend, which are represented by LMB. As you recall, in order to make a forecast, we find L plus H times B, but since this is just one step ahead, H equals one. And so the value of our prediction is just L plus B or more specifically L at T minus one, plus B, A, T minus one. So that concludes how to update the level. It is still exponential smoothing on the input signal way T using L of T minus one A plus B of T minus one as the previous smoothed value. Finally, we have the trend equation, as you can see, the general form of this equation still follows that of exponential smoothing. We have better times, something plus one minus BITA times the old value. In other words, the trend to B of T is also an exponentially smooth estimate. But what is it an exponentially smooth estimate of? Well, that’s the thing that goes in front of the beta. So what goes there? In fact, it’s just elev T minus elev T minus one. So why does this make sense? Remember that B of T is trying to estimate the slope or the trend of the signal. The slope of the signal is the value at one time, minus the value at some other time divided by the difference in time. In our case, the time difference is just one, because we’re updating B of T on every timestep. So of T minus elev T minus one makes sense. You might think y ltt and not Y of T remember that one way of thinking of a time series signal is that it is noisy. The level of T represents the smooth version of that signal without the noise. And so it’s making the difference in L.A. is a better estimate of the overall trend because it essentially removes that noise from the equation. Recall that previously Alpha was treated as sort of a hyper parameter that could be tuned depending on what you were trying to do. In fact, this is often the case. For example, suppose you’re working as an audio engineer and you would like to remove some noise from a sound file. Well, if you have experience with sound editing, then you probably know that you often have to choose some parameters yourself. You tune these to your tastes depending on what the final output signal sounds like. You want it to sound subjectively good, which in general is not something that can be quantified. On the other hand, with Holtze linear trend model, now we’re getting closer to the machine learning picture where what we care about is predictive accuracy, we want our forecast to be good. And now that our model can exhibit a trend, it actually has a chance of making decent predictions. Therefore, the parameters that we learned about in this lecture, Alpha and Beta will no longer be chosen arbitrarily, but rather we can fit them as usual. If you’re familiar with machine learning, then you know, this involves setting up a lost function like the mean squared error and then minimizing the squarer over the training set using methods such as gradient descent. If you are unfamiliar with this process, then you don’t need to worry because stats models is going to do all this work for us. Finally, let’s take a quick look at how the code will look at a high level as before we start by instantiating the model, unlike cyclosarin and most modern machine learning APIs, this is where we pass in the data, which is a univariate time series. Unlike Saikat learn, since the data is UNIVARIATE, it’s OK if this data array is one dimensional. Next, we call the fit function, which in this case doesn’t take in any parameters inside yet learn. This is where the data would usually go. This returns a result object we can use the result object to obtain the predicted values from the set by calling the attribute fitted values. And if we want to forecast, we can call results forecast passing in the number of time steps to forecast.

12. Holt’s Linear Trend Model (Code)

In this lecture, we are going to apply Holtze linear trend model in code. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So first, we are going to import the whole class from stat’s models. Next, we’re going to instantiate a whole to object passing in our data frame. Next, we’re going to call the fit function to fit our model. As you know, this returns a results object next. We’re going to assign the fitted values to the whole column in our data frame. Next, we’re going to plot the whole column along with the original Time series. As you can see, this is similar to the previous model where all it does is seem to track the previous value. In the next block, I’m going to again remind you what not to do, which is to shift the fitted values backwards so that they look like they match up perfectly. It may seem tempting, but remember, you have to be consistent. So if later we see a model that actually does fit nicely, you still have to shift it backwards and it’s going to look very strange. The next step is to use our train to split again so we can do a real forecast, so let’s create a new holds object and pass in the train set. Then we call a fit function with no arguments. Next, we assign the fitted values to our data frame for the rows corresponding to the train set. Next, we call forecast for NPS test time steps, and we assign the result to our data frame for the rows corresponding to the test set. Now, when we plot the whole column, along with the original Time series, we can see both the end sample predictions along with the out of sample forecast. As expected, we get a straight line at trending upwards, which is exactly what the whole linear trend model should it do in this scenario.

13. Holt-Winters (Theory)

So far, we’ve gone through quite a few steps from a simple moving average to the exponentially weighted moving average to viewing the moving average as a predictive model and finally adding a trend component in this lecture, we are going to complete this journey with the full winter’s model. The whole winters’ model extends the whole linear trend model by adding a seasonal component. This makes sense, especially for our data set, because there is a clear seasonal pattern. In fact, for many time series data sets, there is a clear seasonal pattern. Think of something like air conditioner sales. Obviously, more people buy air conditioners in the summer than in the winter. We have all kinds of sales that are seasonal, such as back to school sales, Boxing Day sales and nowadays Black Friday sales, which I’m sure you’re all familiar with. So how does the whole winter’s model work? The basic idea is that for the kinds of Time series we would like to model, there are generally three components. There is the trend component, which we’ve seen. There is the seasonal component, which we are just now seeing, and there is the level component, which is sort of the average value around which everything else fluctuates. These three combine together to form the full time series signal. The question is how do we combine them together, in fact, for the whole winter’s model? There are two ways of doing this. The first method is called the additive method for this model, the seasonal component or the periodic component is simply added to the trend and the level. So you can imagine a noisy line added to some periodic signal. And this periodic signal being periodic does not change over time. The second method is called the multiplicative method for this model. The seasonal component changes proportional to the level of the Time series. This seems like it might be a good fit for our data set, because it seems like the wavy part actually increases in amplitude as the level gets larger and larger. So let’s look at the additive method in component form. As usual, I think the best way to understand these models is to pay careful attention to each of these equations. The forecast equation is sort of the centerpiece of the model because it tells you how each of the components combine to give you the full output. So what does the forecast equation look like here? Well, first, let’s recognize that two thirds of this equation we’ve already seen this is just holds linear trend model. It says that for each step that H increases the non seasonal part of the forecast increases by B of T. The new part is the S term, which obviously corresponds to the seasonal component. Now, you might find these indices very confusing at first, but we’ll go through them slowly so you understand what they mean. First, we need to define this new variable M, which is used to define the period of the seasonal component. Some people refer to this as the frequency, but it’s more accurate to think of it as the period. Remember, that period refers to the length of time that it takes for a signal to repeat itself. For example, the period of the sine function and the cosine function is too high. Frequency is actually inversely related to the period. In any case, this is one of those variables that must be chosen manually. But in real world situations, it probably won’t be that hard. As an example, if you’re looking at sales data over each month, then we might expect the period to be 12 since there are certain events that happen every year which affect sales. Obviously, this will require some domain knowledge. So the data science practitioner, that’s you should apply what you know about the data to choose this value correctly. So now we know what M is. We also know what T and since we’ve seen them before, T is just the last known time for which we have real observations and the number of steps ahead that we are forecasting. But what is K? Let’s ignore this strange formula for now. Intuitively, K is a number chosen so that we look back the appropriate number of steps in our training data to choose the correct seasonal component. For example, suppose that our data goes up until December twenty twenty and the frequency of the data is monthly. Suppose that we would like to make a forecast for March twenty twenty one. In this case, the training data is from December twenty twenty and any time before that to get the seasonal component for the March twenty twenty one forecast, we would like to use the seasonal component for the latest month of March that we know of, which is March twenty twenty. We would not want to have, say, the seasonal component from November twenty twenty because that’s an entirely different month, which includes Black Friday, which definitely includes data we don’t care about in March. On the other hand, note that the level L and the trend B both come from time t, the latest measurement, only the seasonal component looks further back in time to grab the appropriate point over the entire season. If you’re mathematically inclined, then we can do an example to make sure that this works, if you are not mathematically inclined, then just understand that intuition and that’s OK. Remember that we’re going to be using a library anyway, so let’s suppose that we have it. T equals thirty six and our data is monthly, so M equals 12. Suppose that we want to forecast 15 steps into the future. So H is equal to 15. So if one refers to January then 12 refers to December as those twenty four and thirty six. So our training data is three years from January to December. Three years later, we want to predict 15 months into the future, which is March, but it’s the March. One year after the latest measurement, we know intuitively that the seasonal component should come from T equals twenty seven Y, because that’s the last known March measurement that we have. In other words, the one that’s closest to thirty six. So let’s see if our formula works, we have equal to the floor of each minus one over M plus one, if we plug in all of our numbers, we get K equal to the floor of 14 over 12 plus one, which is all equal to two. Therefore, the time index for S will be thirty six plus fifteen minus twelve times two, which is equal to twenty seven. And thus we have confirmed that this strange formula does indeed work. All right, so now that we understand the forecast equation, the others are pretty simple. Next, we have the legal equation. This is almost the same as the linear trend model. And you should feel comfort about the fact that it’s the same old exponential moving average. The main difference is that we subtract the seasonal component from Y note that we subtract the previous seasonal component at index T minus M. This makes sense since in our equations generally, why is considered the forecast. So it’s always ahead of all the other components. Next, we have the trend equation. This one is exactly the same as before. It’s an exponential moving average of the trend as measured by the difference in levels. Finally, we have the seasonal equation, this one might seem strange at first, but at least you can recognize that this, again, is an exponential moving average in front of Gammer. We have the new value and in front of one minus Gamma, we have the old value. Again, recognize that the old value comes from one period ago rather than one timestep ago. So that’s why the previous S is indexed by T minus. For the new value, this makes sense when you consider the forecast recall that Y is equal to the level plus trend plus seasonal component. Therefore the seasonal component is Y minus the level, minus the trend. If you think of it like an equation, you’re just moving the level in the trend to the other side to isolate the seasonal part. All right, so now that you understand the additive model, let’s move on to the multiplicative model. Since you already understand how the additive model works, it’s not that much effort to understand the multiplicative model as before. Let’s start with understanding the forecast equation based on the name of the model. I think the forecast equation makes intuitive sense instead of adding the seasonal component. We now take the linear part and multiply it by the seasonal component. The general equation at a high level can be thought of as Y equals L plus B all times s for the level update. We now have Y divided by S. That makes sense because before we had Y minus S. For the trend equation, there is no change for the seasonal equation. We have Y divided by L plus B that makes sense because before we had a Y minus O plus B. All right, so as with our previous lectures, now that we understand the theory, we’re going to discuss the main parts of the code. So first, we’ll need a new class, which is simply called exponential smoothing. This might be a little confusing since one might think of the Ummah as exponential smoothing. But just remember that in this library, we qualify it by saying it’s simple exponential smoothing. So not simple. Exponential smoothing refers to whole winters’. Next, we instantiate our model in the constructor, we pass in the data, and we also specify whether the trend and seasonal components are additive or multiplicative. We also specify the period which in our case will be 12 since our data is monthly. Note that in this lecture we discussed two models with a seasonal component was either additive or multiplicative in both these cases. The trend was additive. Stat’s models also gives you the option of making the trend a multiplicative if you think that fits your data. So that might apply when your trend curves rather than going in a straight line. The next step is to call the fit function, which does not require any parameters. And as usual, this returns. A whole winters’ results object from which we can then obtain the fitted values as well as call the forecast function.

14. Holt-Winters (Code)

In this lecture, we are going to apply the whole winter’s exponential smoothing model in code. This is our final model for this group of models, the culmination of our study of exponential smoothing. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Let’s start by importing the exponential smoothing class. Next, we’re going to create an instance of our model. This time we’re not going to bother fitting a model to the entire data set, but rather we’re going to skip right away to the train to split. So the first variation we’re going to try is the additive trend and additive seasonality, we set the seasonal periods, the 12, since we know that the data cycles yearly and the frequency of the data is monthly. Next, we call the fit function. So let’s run this. Now, due to numerical issues, we get a convergence warning, however, you’ll see that the model has been fitted appropriately. Next, we assign the fitted values to the whole Winters’ column of our original data frame for the train rose. Next, we call the forecast function for and test the steps in, assign the results to our original data frame for the tests Rose. Next, we plot the whole winter’s column along with the original data set. So this is very encouraging, unlike the simple exponential smoothing model and the whole linear trend model. This model fits very well for both a train and test notice, importantly, that the predictions are not lagging behind the Time series. So hopefully now you are convinced that it is not correct to shift it backwards by one step. And in fact, the lagging is only due to model mis specification. Remember that you have to be consistent. If you shift back for one model, you have to shift back for all of them. And for this model, that would be very silly because then the predictions would be ahead of the data, which doesn’t make any sense. Now that we know our model fits decently well, it would be a good idea to calculate some metrics. Although there are many metrics which are used in time series analysis, the root mean squared error, we’ll be just fine for us. This metric makes more sense, in my opinion, than something like the absolute error, because these models actually minimize the squared error directly. Therefore, it would make less sense to check the absolute error because we didn’t optimize for that in the first place. If you’re going to check the absolute error, then you may as well make the absolute error. Your lost function in the root mean square. There is just the square root of the square there, which puts it on the same scale as the original data. So minimizing the mean squared error is equivalent to minimizing the root mean squared error. So here’s how we calculate it. We take why the predictions and t the targets, although it doesn’t really matter which is which, and then we subtract them. This does an element y subtraction. Next we square the result. This is also an element y’s operation. Next we call the mean. So now we have the means squared error. Finally we take the square root so we get the root mean squared error or messy. Now, just out of curiosity, I would also like to calculate the mean absolute error. This calculation is a little simpler. We just take the difference between why and t take the absolute value and then take the mean, hence mean absolute error. Next, we check the arms on both the train and test set, as you recall, the train predictions are stored in the fitted values attribute and the test predictions are made using the forecast function. As you can see, I’ve already switched around, which is the prediction and which is the target, since it doesn’t really matter, which is why and which is t you will get the same answer either way. All right, so the train IMC is about 11 something and the test pharmacy is about 15 something. When we checked the mean absolute air, we get about eight something for train and a 10 something for a test. Next, we’re going to try the whole Winters’ model again, but with different parameters, recall that it seems that the amplitude of the cycle increases with the level of the Time series. This suggests that a multiplicative model may fit better. So for this model, we’re going to have an additive trend and a multiplicative seasonality. Again, we call it a sign the trend predictions and the test predictions to the data frame and then plot the result. So we’re just doing it all in one step, since you already know how each of these steps works. So from my perspective, at least, this model appears to fit better than the purely additive model. But let’s check our accuracy metrics. So when we check the arms, we get about 10 something for the train set and a 17 something for the TSA. This is a better train error than the purely additive model, but the test error is worse. Therefore, we would not want to choose this model. If we check the mean absolute air, we see the same thing, the trainer is better, but the test error is worse. Next, let’s try Hault Winters’ again, but with the multiplicative trend and the multiplicative seasonality, this kind of makes sense because the trend and the Time series is not exactly a straight line. The growth actually seems to be accelerating over time. Again, we call it a sign, the predictions to our data frame and plot the result. It’s difficult to tell whether this is a better or worse fit, so let’s check our accuracy metrics. All right, so here we see an interesting pattern, the train arm is about nine something and the test arm AC is about 25 something. So the trainer is better, but the test there is much worse. We see the same thing for the mean absolute error, the train error has gone down, but the test there has gone up to about 20 something. Therefore, while this Time series may look like a multiplicative model is a better fit, it actually overfit to the training data and we get a worse result.

15. Autoregressive Models - AR(p)

In this next part of the course, we are going to begin discussing a different kind of time series model, which is more like the machine learning type of model that you might be familiar with. In particular, we’ll be looking at auto regressive models, moving average models and combinations thereof. Generally, these are known as Arima models. R stands for auto regressive, Emmas stands for Moving Average and I stands for Integrated. You will learn what all of these terms mean in the coming lectures. Note that in this context, moving average does not mean what it meant in the previous lectures, so don’t get them confused. The names are similar, but as you’ll see, the actual models are completely different. This lecture will focus on auto regressive models specifically. But first, what is the difference between a Rhema models and the exponential smoothing models we previously discussed? As you recall, the exponential smoothing models are built for very specific kinds of data. They model certain aspects of Time series very explicitly. In particular, they model linear trends and they model seasonality. These are built into these models. On the other hand, you’ll find that Arima models impose no such structure in that way. They are more in the spirit of modern machine learning, where you take a model and you try to fit it to your data, whatever structure your data may have. So what are auto regressive models, auto regressive models are basically linear regression models where the inputs also known as the predictors, are past data points in the Time series. Let’s first review how linear regression works since we may not have gone into enough detail earlier in the course. As you recall, the simplest linear regression model looks like this. We say Y hat equals M, X plus B, in this case, X is the input and Y hat is the prediction and B are parameters that are found through minimizing the error of the predictions. One simple example of linear regression would be trying to predict salary from years of experience. So X would represent years of experience and Y would represent salary. But what if we have more than one input, you might think years of experience might not be good enough on its own to make a good prediction about salary. Let’s say we want to take into account age as well. Now, our model will look like this y hat equals one time zwaan plus W two times X two plus B in this case, X one could represent years of experience and X to represent age as before. One, two and B are found by minimizing the error of the predictions compared to their true values in the data. Let’s think about what the interpretation of one and two and B could be as before B represents the Y intercept, that is the value of Y when all the X’s are zero. If you plug in X one equals zero and X two equals zero, you can see that this is true. The WS, on the other hand, tell us how each of the X’s affects Y if X one increases by one and everything else that is X two remains constant, then Y hat will increase by one. If you don’t see this right away, I would recommend plugging in some numbers until you are convinced. So one is the amount that we had increases by when X one increases by one, to give you a concrete example, if X one represents years of experience and one equals 5000, then that means in our model, every additional year of experience should lead to a five thousand dollar increase in salary. On average. Note that we sometimes call this model multiple linear regression due to the fact that there are multiple inputs, whereas before we would call that simple linear regression. Usually I don’t make this distinction and I just call it linear regression. OK, so what does this have to do with Time series and auto regressive models in particular? Well, an auto regressive model is nothing but a multiple linear regression model where the inputs are the past values. In the Time series, for example, we had a time t is a linear function of Y a time T minus one Y a time T minus two and so on up to Y a time T minus P. Note that when we use past data points in the Time series, that is why it time T minus one back to Y time T minus P. We call this an AARP model. Moreover, mostly we say that this is an auto regressive model of order P. Note that an equivalent way of expressing a linear regression model is this, instead of having Y hat on the left, we just have Y at time T on the right side, we have added a noise term for time t epsilon sub t. What this says is that what do we actually measure that is the true Y is a linear model of the inputs plus some noise. This must be the case since as you’ve seen, usually when we do a linear regression, the data points do not fall exactly on the line. Another way of thinking about this is that y had AT&T is just the expected value of Y of T assuming that the noise terms expected value is zero. Now, just as a theoretical exercise, we are going to discuss how you might use psychic learn to implement an auto regressive model. In actuality, we are going to use stat’s models, but I find it very instructive to think about how you use it, learn which forces you to think about the structure of the data. Let’s think about what a data set might look like for a normal linear regression model where I’m trying to predict salary from years of experience and age. In order to do this, I might do a survey where I ask everyone in the office to anonymously fill in a spreadsheet that I’ve created. After all of my coworkers have filled in the spreadsheet, I will have something that looks like what you see here. Each row corresponds to a different person. Each column represents an input feature in our case. That’s years of experience and age. We call this big table X. In fact, it is indeed a table, but it is also a matrix. Furthermore, we have Y, which is just a single column of salaries. Obviously each of these salaries corresponds to the same row in the X matrix. So if the first row of X belongs to Alice, then the first row of Y is Alice’s salary. Using this knowledge, can we build a linear regression data set for an AP model? The answer is yes. Well, I’m just going to show you the answer, and your job will be to check that this makes sense. In fact, you could even implement this yourself using Scicluna as an exercise. So the idea is this. Suppose that we start with the fact that there are 10 data points. Why one up to why ten? We know that the equation to predict Y four is a linear function of Y one way too. And Y three. Therefore, if Y four is the target then y one y two and Y three are the inputs. Similarly, if Y five is the target then Y to y three and Y four are the inputs. If we keep doing this, we get the table that you see here. Note that in machine learning, we typically say that the size of X is envied where and is the number of samples and it is the number of input features for arena models. We will stick with the convention that the number of predictors is P. So in case you are confused about this, just remember that in the context of a Remar D is equal to P. So at this point, you know how to convert your time series into a tabular data format that can be passed in the cycle, learn once you have this, the rest is easy. As usual, you can create a model of type of linear regression you call model that fit to fit your model. After that, you can call model that predict to make predictions or model that score to check how good your model is. This is just another instance of my rule. All data is the same. It doesn’t matter what your data is. The basic API never changes whether you’re predicting salary or you’re predicting a time series. Another interesting topic to consider is this we know based on our knowledge of machine learning, that linear models are not that powerful. In fact, they can only represent lines or planes. In addition to my rule, all data is the same. I have another rule. All machine learning interfaces are the same. What does this mean? It means that most of the time when you were doing machine learning, you can use different models say was learn without any change to the rest of your code. Previously we used linear regression and we noted that this is an auto regressive model. But why should we be limited to just linear regression? What’s stopping us from, say, using a neural network or random forest, which we know are much more powerful models? If we have our X and we have a Y, there was nearly no change to the code required. Simply replace the linear regression class with some other class. On the other hand, it’s also important to remember that in this class, which is focused around financial data, there was a strong emphasis on the modeling aspect rather than predictive accuracy. What you will learn in the following lectures is that Arima models allow us to understand our data in a much more in-depth way than simply plugging in a neural network. Yet another way to understand auto regressive models and Arima in general is this, as you know, one of the major themes of this course is price simulation. What were you actually studying in? This course is random processes and naturally a random process can be simulated using a computer. We looked at a few examples in previous lectures, such as simulating random walks and simulating a sequence of coin flips to decide whether you would take a step to the left or take a step to the right. So it’s useful to ask the question if we are given an AP model that is all of the weights of the model, how do we generate a time series? Not that this is kind of the opposite question that we are used to considering in machine learning and machine learning. The question we usually ask is, given the time series, tell me the weights. Now we are going in reverse order. Given the weights, tell me what the time series might look like. So how can we do this, let’s suppose we have an R three model, so the next value in our Time series always depends on the last three values. Let’s suppose also that the first three values are given so that the first value we are responsible for generating is Y for in this case, the first step will be to generate Epsilon for the error at time four, which is a normal with mean zero and some variance sigma squared. Then we can use our formula to calculate Y for next. We have to calculate Y five. Before we can do that, we first generate Epsilon five again from the same normal with mean zero and variance sigma squared. Then we add Epsilon five to the Y to Y three and newly generated Y four terms and that gives us Y five. Then we generate Epsilon six and Y six, epsilon seven and Y seven and so on and so forth. So you see that by doing this we understand that this model is more expressive than a simple random walk. This would be a random walk if we only had a one previous value and a weight of one. Instead, we have a bunch more terms and arbitrary weights. In the next few lectures, we will make this model even more expressive.

16. Moving Average Models - MA(q)

In this lecture, we are going to discuss another integral part of the Arima model, the model, as you know, M.A stands for moving average. However, it’s important not to conflate this with the simple moving average and the exponentially weighted moving average we discussed earlier in this course. As you will see shortly, this model is very different from those. So let’s get right to what the moving average model looks like. The moving average model is similar to the auto regressive model in that it is a linear function of something, but what is it a linear function of? In fact, it is a linear function of past error terms. This should definitely strike you as odd in pretty much all of machine learning. We typically create models which depend on input data in this moving average model. There is no input data to be seen. Instead, the Time series depends only on errors. Of course, in order to know these errors, we must have compared to the previous predictions with the previous values of the Time series. Note that our abbreviated form for a moving average model with autocue is Macu, this means that the output y depends on cue passed error terms in addition to the latest error term epsilon t. Here’s one way to think of the moving average model that might help you rationalize its name. Consider what the expected value of Y of T should be. As you know, we treat errors as normal’s with mean zero and variance sigma squared. Well, if we take the expected value of Y of T, we simply get C, since the expected value of each of the errors is zero. Hence we can think of the bias term C as the average value and then each of the errors as fluctuations that make Y have to go up or down around. Another way to think about the moving average model is in terms of simulation. In fact, simulating a moving average process is somewhat easier than an auto regressive process, let’s suppose that we want to simulate and may to process so the output y depends on to past error terms in addition to the current error term. First, we need to generate Epsilon one and Epsilon two. These are just samples from some normal with mean zero and variance sigma squared. Then we generate Epsilon three, then we calculate Y three according to our formula, which depends on Epsilon one, two and three, as well as the model weights which we assume have been given. Next, we generate Epsilon four. Then we calculate Y four, which depends on Epsilon two, three and four. Then we generate Epsilon five and Y five, epsilon six and Y six and so on. So as you see, this process really amounts to nothing but generating samples from the normal and adding them together. Therefore, you would use this model if you think that nature actually behaves this way to generate the data which you are observing.

17. ARIMA

In this lecture, we are going to combine what we learned about in the previous lectures and build up the full Arima model. Before we do that, however, let’s first discuss what we get when we combine the R and the M models only, which gives us the AMA model. After that, we’ll discuss what the AI component means and then combine that with the AMA model to give us the Arima model. So I think the AMA model is pretty straightforward once you understand the auto of aggressive and moving average parts separately, the AMA model simply adds them together. Therefore, you would use this model if you believe that each point in the time series is linearly correlated with both pass points in the Time series, as well as past errors of the model. Note that the abbreviated form of this model is ama p. Q This is short for auto regressive moving average model with the auto regressive part has auto P and the moving average part has autocue. Now that we know what Amah is, let’s talk about Arima, the IPART part in Arima stands for Integrated. Therefore, the full name for this model is auto regressive, integrated, moving average. What is interesting about this model is that the integrated part has no parameters. So the way it looks is very different from the auto regressive and moving average parts. So how does the integrated part where. To understand the integrated model, we have to understand differences, I suppose that we have some time series Y of T to perform differences on this Time series, you would define a new time series, Delta Y of T, which is defined as Y of T, minus Y of T minus one. That is for each point in the Time series we subtract of the previous data point. Now a good question to ask is, why would you want to do such a thing? In fact, this is not the first time we have seen differences in this course. We have seen it twice previously. One time we saw different. Same is with log prices. When we perform different single log prices, we get the log return. As you know, this is a pretty important quantity in finance. Another time we saw different is what the whole winters’ model, the first difference of the level is used to estimate the trend of the series. In other words, one way to think of differences is defending, or in other words, separating out the trend. In fact, we see this in Stock Price Time series as well. After taking the first difference of the log prices, we get the log return, which is generally a noisy signal that fluctuates around a value close to zero. So why do we want to difference our Time series, the reason for this is when we use Arma models, that is auto regressive, moving average models. We want the time series. We use the model on to be close to stationary. Now, what does stationary mean? Again, at a high level? It means that the distribution of your signal does not change over time. There are more details we can discuss regarding stationary, but we will leave this for a later lecture. Stationary is a nice property because it means that various statistics, such as the mean variance and auto correlation, will remain constant over time. This is a good thing when you’re trying to fit an arm, a model, because remember that each window of the Time series is like another training point when you are fitting the model. Now, this slide is optional, but I want to elaborate on the previous point a bit more. Recall that earlier I talked about training your own auto regressive model, using Saikat. Learn by first building your data set from the Time series. As you can see, this data set is built out of sliding windows from the original time series. First, the window covers Y one y two and why three. Then we slide it over one step. Now the window covers y two y three and Y four, then we slide it over one step again and so on. One important aspect of machine learning is that the data in your data set is assumed to all come from the same distribution. If it didn’t, then you wouldn’t be able to learn anything. Remember, machine learning is pattern recognition and pattern recognition is machine learning. If we mess up the pattern, then we mess up the machine learning model. Imagine, for example, that we had our table of salary data. Suppose that we work at a software engineering company and all of the coworkers who helped you fill in your spreadsheet were software engineers. Then all of a sudden, one of your buddies from the marketing department fills in their data on the spreadsheet. But unfortunately, marketers get paid at differently than software engineers. And so it’s likely that the distribution for marketers salaries is different from the distribution for a software engineer salaries. Your model might see this data point as an outlier, which might bias the results. So that’s why we want all of our data to come from the same distribution. Otherwise, what we are trying to learn won’t make much sense if we’re trying to predict software engineer salaries. We don’t want data from people who are not software engineers. For us, we have a time series and each position on this Time series makes up a new data point in our training set. If we would like all these data points to come from the same distribution, then the Time series being stationary accomplishes that because that’s exactly what stationary means. So let’s get back to stationary now, we know why stationary is good. Well, it turns out that different seeing a time series often helps to make the Time series stationary. Now, remember that when you’re working with real data, nothing is exact and you’re always approximating. Yes, we know about volatility clustering. So we know that the variance of the log return can fluctuate over time. Does that mean we throw out a rhema completely? The answer is no. And sometimes we may even find that we have to difference twice in order to get a time series that looks somewhat stationary. Usually, however, we don’t difference more than twice. OK, so what does defensing have to do with a Arima? We say that a process is ID if it is stationary after being differenced D times and ID process is a process that is integrated to order D. Once we have this, we can define the Arima model. The Arima model has three arguments P and Q, as you recall, P refers to the auto regressive part. Q refers to the moving average part and now we know that D refers to the integrated part. Thus Arima P. Q is just a model where we have difference. The original time series D times before applying the ama p q model. So that’s why the eye part of the Arima is strange. Unlike the auto regressive and moving average parts which have some formula that you can use to make predictions, the integrated part describes an operation that you perform on the data. Now that we know what the full Arima model is, we can see that all of the previous models we’ve discussed are actually just special cases of the Arima. For example, a remap zero zero isn’t IRP process. This is also the same as an arm up zero. Similarly, Arima zero zero Q is an AMA zero. Q Which is also an Macu Arima zero zero is an ID. One interesting special case is Arima zero one zero or equivalently, I one, this is what is known as a random walk, which we discussed previously. This model looks as follows. It says that Delta Wafty is equal to Epsilon of t the noise at time t. Why is this a special case? Well, it just means that there is no auto regressive part and no moving average part by omitting these two. All that we are left with is noise, the left side delta. It is just the difference. That Time series, equivalently it says that Y of T minus Y of T minus one is equal to the noise at time. T if we move the Y of T minus one it to the other side we get that wave T is equal to Y of T minus one plus Ypsilanti, which is exactly the random walk formula we discussed earlier. It says that whatever process in nature leads to our Time series, it takes the existing value Y of T minus one, generates some random noise Ypsilanti and adds them together to get the next data point y. T. Another way to view this, if we think about financial quantities is this suppose our data set is a time series of log prices, then the first difference is the log return. A random walk says the log return is just Ypsilanti, the noise. What this tells us is that there was nothing about the log return that is predictable except for the expected value of the noise. It does not depend on any past values, which is what an auto regressive model would tell us. It does not depend on any past noise terms, which is what a moving average model would tell us. Instead, a random walk says that the log return is purely noise from a single point in time and hence completely unpredictable. Aside from the mean of the noise, therefore, if we later fit in a rhema model to a stock price time series and we find that the best RMI model is Arima zero one zero, this tells us that the data follows a random walk and the return can’t be predicted using previous values in the Times series.

18. ARIMA in Code (pt 1)

In this lecture, we are going to look at how to use a in code for the first few Arima lectures will continue looking at the airline passengers data set to build your intuition on how Arima works. And then to cap off this section, we will return to looking at stocks. So in this lecture, we will focus solely on airline passengers. Since using this data set, it will be much easier to understand the behavior of a remar under different and queue values with stocks. This is kind of difficult because you can’t really tell if there’s a problem due to bad settings or if it’s simply due to the inherent randomness in stock prices themselves. So hopefully that makes sense for you. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So we’re going to start by downloading our airline passengers CSFI. Next, we import Pendas lumpia matplotlib. Next, we’re going to load in our CSFI using to read CSFI. Next, we do have got head just to remind you of what’s in our data frame. Next, we do a dot plot to remind you what our data set looks like. Next, we’re going to calculate the first difference in our data. Note that previously we used the shift function and the subtract operation manually. However, this can also be done by simply calling the diff function. As noted earlier, lots of panners functionality has been built explicitly for statistical time, series, analysis and finance. So it’s not surprising that such a function exists. Next, we do the player so you can see what the first difference of the Time series looks like, as you can see, it’s not quite stationary because there is some seasonality and the variation is increasing over time. Next, we’re going to compute the log of the passengers column by calling airport log on the passengers column. Next, we plot the log, passengers call them. Notice something interesting about this, which is that the values don’t seem to be growing as rapidly as they were in the non logged time series. Furthermore, the amplitude of the cycles looks more constant than it was before. Next, we’re going to import the Arima class from stat’s models. Next, we’re going to set the frequency of our data from index to months. And we’re also going to split our data into train and test using the last 12 months of our data set, as the test said. Next, we’re going to test in air one model. This is a model in which the current value in the Time series depends linearly on just the previous value, as you recall. This is equivalent to a rhema one zero zero. Next, we call a treatment outfit, as with hot winters, this returns and a remote result object, unlike psychic learn, this does not return a model. Next, we call a rhema result predict to obtain the training predictions will place this in a new column in our data frame called R-1. Next will plot the passengers column along with the air one column to see how well our model fits with the data. So as you can see, we get this kind of delayed behavior, which should remind you of what happened with whole winters, as you recall, this is how hot winters also behaved when the model was specified. In these cases, it seems like the best your model can do is to just copy the previous value or close to it. Next, we’re going to grab our out of sample predictions by calling Arima results forecast for any test steps. Note that this returns multiple things. Unlike the whole winters’ model, it returns the forecast, obviously, along with the standard error and the confidence intervals. We’ll see how we can make use of these confidence intervals very shortly. But for now, let’s just plot the air one column again against the passengers data. As you can see, this model is not very good, in fact, it doesn’t even capture the fact that the data is trending upwards. Instead, the forecast is going in completely the wrong direction. So even a naive forecast would be better than this. Next, we’re going to write a function that will plot the fitted values of an arena model along with the forecast and confidence intervals. This will be helpful for the rest of this script so that we don’t have to keep writing this code over and over again. So this function, it takes in one argument, which is NRMA result object inside the function. The first thing we do is set the size of the plot, which will be 10 columns and five rows. Next, we call the plot function passing in the passengers column of the data frame. This will plot the true data. Next, we’re going to obtain the train predictions by calling a result that fitted values. Then we will plot the train predictions by calling the plot function again, passing in the train index along with the train predictions we just obtained. We’ll make this plot green and we’ll give it the label fitted. Next, we will plot the forecast to start, we need to call results forecast for any test steps. This gives us back the forecast itself, the standard error and the confidence interval. Next, we will call the plot function again, this time passing in the test index along with the forecast values. We’ll give this the label forecast next in order to plot the confidence bounce. We’ll use the function fill between the arguments to this will be the test index. Once again, the confidence interval, return value at zero, the confidence return value at column one, the color red and alpha equals zero point three. To set the transparency, if you want to double check that the confidence interval return value is actually formatted this way, you’re encouraged to check the documentation. Lastly, we call a legend so that the legend will show up in our PLI. All right, so in this next block, we’re going to test the function we just created passing in the Arima result object, we got back earlier. OK, so this is our Awana forecast again, not very good, but we have many options to try next. Next, we’re going to try something which I think should be pretty natural for most machine learning people, if our model is bad, why not simply add more inputs? And so in this next block, we’re going to use an Artan model. Luckily, there’s not much more work to do beyond what we’ve already done. We start by creating another Arima object with the order at 10 00. Next, we call a remote outfit and then we pass the Arima results into our plotting function. So let’s run this. OK, and we see that this does, in fact, do a much better job. The first thing we notice is that it does not have the Lagan characteristic that the previous R-1 model had. This suggests that the model is actually learning to anticipate the pattern rather than just copying the last value. If we look at the forecast, we see that the model really does learn that the signal is periodic. Unfortunately, it seems to underestimate the true value significantly. Next, we’re going to test and may one model, as you recall, this is equivalent to a rhema zero zero one as before. We can just repeat the same code from earlier. So let’s run this. OK, so this looks pretty bad. It’s clear that the May one model does not work. In fact, it’s even worse than one. As you recall, the moving average models expected forecast is always a constant value. So intuitively, we know that this is probably not the right choice. All right, so next, we’re going to investigate using the log passengers. Let’s begin by taking the first difference of the log passengers. Again, we use the function. Next, we plot the first difference of the log passengers. As you can see, the behavior is indeed what we expected. Unlike the first difference of the non logged passengers, these values do not seem to grow that much over time. All right, so next, before we do anything with the log passengers, I want to fit a new model on the non logs passengers. We know that difference is good because it at least seems to remove the trend. So for this model, I’m going to use a rhema eight one one I’ve chosen P equals eight since it seems like regression on more pass values helps. I’ve chosen D equals one since as we saw, this removes the trend from the Time series and I’ve chosen Q equals one for good measure just so that we have a full Arima model with all the components. Now in order to plot the results from this Arima, we’re going to need to create a new function y. The reason why is because we’ve differenced our data, as you recall, when we difference our data. This removes one row from our data set, specifically the first row. Each point in the different time series is the current value minus the last value. But for the first row there is no last value and therefore that value can’t exist. You’ll see that if you try to use the previous function that we wrote for plotting the results, you will get an error. You were strongly encouraged to try it yourself and to see what the error actually is. OK, so this function is called plot fit and forecast int. It now takes in a three arguments instead of just one. First, it takes in the Arima results object, same as before. Second, it takes in a value for D. D will represent the first row in the Difference Time series that actually exists. Third, we have an argument called Call, which will tell us which column of the data frame we fit, the model line that will either be passengers or log passengers. Of course, we have to choose the appropriate one in order to plot the correct data. With the model predictions inside the function, we first plot the data. This looks the same as before, except now we index the data frame using the call argument rather than a fixed column name. Next, we grab our training predictions using the predict function. I’ve used the predict function so that we can be explicit about our indices, but also because it’s necessary in order to get the values back in terms of the original time series rather than the difference the Time series. Notice that for the start argument I’ve passed in train that index index at D before this value would have just been zero instead of D. Also, notice this third argument type for this argument, if we pass in the string levels, this will return the model predictions in terms of the original time series rather than the difference this Time series. Remember that internally we are actually fitting an arm, a model on the Difference Time series. So the type argument, instead of predicting the difference Time series actually accumulates back to the original level. Next, we call the plot function for the predictions again, notice that for the index, I’m only passing in the values from the death row onward. Next, we plot the forecast, this part is the same as before we call the forecast function for n test timestamps, then we call the plot function to plot the forecast. Then we call the fill between function to plot the confidence interval. Lastly, we call the legend function to show the legend. Next, we’re going to call our function, which will plot the results. OK, so as you can see, this does pretty well better than a purely auto regressive model, the train fit is pretty good and the forecast captures the seasonality, although it seems to underestimate the peak and overestimate the trough. Next, what I want to do is create a function at that plotts, the fit of our model on the Differenced series, as you know, this is what Stat’s models is doing internally in order to show us the final forecast. It’s actually taking the predictions on the Differenced series and then accumulating them to show us the final non differenced forecast. So this plot is something you might want to look at if you want to know how well the arm, a part of the model fits. And that’s the part of the model that actually contains the parameters. So first, notice that this function, it takes in the same arguments as the previous function are results object, the value of D, which is the offset to skip over the nine values and the column name inside the function we first call predict to get the predictions. Note that this time we do not pass in a type equals levels. Instead, we will leave the tape as the default value which will return whatever actually got fitted with the AMA model after different. Alternatively, you could have just called the attribute fitted values, but I think this is better for understanding what’s happening under the hood. Next, we take the difference in the data frame. Now, if you want it to be really correct, you could difference this at the Times. But I didn’t since we already know that there’s one. Next, we plot the Difference Time series along with the train predictions, then we call a function on our results. Again, this is for RMI eight one one on the non logged passengers. OK, so you can see that these results are pretty good, the data set is not stationary and the variance is increasing over time. However, the model seems to capture this, at least for the sample data. Next, we fit in a Rhema model with the same and values, except this time we use the logged dataset. As usual, we call fit and then we plot the results. All right, so as you can see, our model does pretty well, maybe even better than the previous model on the non logged data set. Next, we call our difference function on the log passengers model. So we see that the model, again, fits pretty well to the Defriended Time series, this time the variance does not increase over time, so maybe the model has an easier time learning. Finally, just out of curiosity, I wanted to fit another model on the log passenger’s data set, this time with P equals 12 and Q equals zero. The reason I chose 12 here is because that’s the length of the period. So it might make sense to use data from all 12 months over the past year in order to make a prediction. OK, so this is the same code as before, except now the Arima order is 12 one zero. So let’s run this. Note that this might take a while since the algorithm used to train Arima is not a closed form equation. OK, so what do we see? Well, it seems like this is our best model so far. Now, in order to really know how well these models have performed, it would be useful to look at a metric such as the root mean squared error. So next, we’re going to define such a function. This will take in at two arguments. The Arima results object and a flag to tell us whether the model was fitted on the log passengers or the non log passengers. Inside the function we call results are forecast to get the forecast. Next, we check is logged. If is logged is true, then we exponentiation forecast to put the data back into the original scale. Finally, we assign the passengers column of the test data frame to a variable called T. Then we assign the forecast to a variable called Y. Then we apply our usual formula for the root mean square error and then we return this value. All right, so in the next block, we compare the root mean squared error for the last three models we created. These are Rhema A1 one on the Raw Data set, RMI eight one one on the log data set and a 12 one zero on the log data set. So let’s run this. OK, so what do we see? Firstly, our suspicions were correct, logging the data first does in fact help the model learn the decrypted series better. Both Arima models have the exact same order, but the version fitted to the log passengers achieves a smaller error. Next, we see that adding more lag’s values to the auto regressive part of the model does really help. RMI 12 one zero achieves the best accuracy compared to the other two by far.

19. Stationarity

In this lecture, we are going to discuss more on the topic of stationery, previously we looked at how different kinds of Arima models might fare on the airline passengers data set. Of course, we were just guessing. It’s not easy to tell what are the right orders to use when fitting NRMA model. In fact, this is the case in general for machine learning in a deep learning. For example, one question I often get from beginners is how do I choose the hyper parameters? How do I choose the learning rate, the hidden layer size, the number of hidden units and so on? I think today people generally have a better understanding. But when I first started my courses, it would make people really angry that there wasn’t some formula that they could use. Or indeed, hyper parameter optimization isn’t a topic for students who like simple direct in a straightforward answers to their problems. Usually it amounts to nothing but a trial and error. As I always say, machine learning is experimentation, not philosophy. If you want to know whether something is going to work or not, well, then you do an experiment. In any case, this idea will come into play later. But for now, what I want to say is this. For Arima, there is a way that you can scientifically and methodically choose your hyper parameters. In our case, these are the orders PD and queue. Note that this process is not exact and does not necessarily lead to the best answer. However, it is statistically sound. The first scenario I would like to consider is stationary. As you recall, this will help us choose the order deep in our Arima model. So this lecture will be split up into two parts, the first part will be a more beginner and practical oriented discussion. We’re going to look at how to determine whether or not a time series is stationary in code. This involves doing a statistical test and checking the P value. The second part will be more advanced and it will discuss stationary in a more exact manner. The second part is optional, so feel free to skip it if you want to jump straight to the code or if you do not like math. So this is the first part on the practical aspects of stationary stationary, loosely speaking, means that the distribution of the data does not change over time. That is, if you look at things like a mean or the variance at any point in the time series, they will always be the same. Looking at when this is not the case is helpful. For example, if you see a time series trending upwards or downwards, then you know that the mean is changing. Therefore, the existence of any trend means that the time series is not stationary. Furthermore, when the variance changes over time, that is also not stationary. So if at the beginning of a time series the value only wiggles around a little bit, but then starts to wiggle around more and more later, and that’s not stationary. We’ve seen this behavior in stock returns which have heteros good activity. So one might consider stock returns to be non stationary, although it is often assumed that they are. So don’t be surprised if we treat stock returns as if they are stationary in the future. Let’s now talk about a practical issue, how can we test whether or not a time series is stationary? Luckily, there is a well known statistical test that does exactly this. It’s called the Augmented Deqi Fuller Test or the eighth test for sure. As we discussed earlier, one way to think of statistical tests is like an API. We have a null hypothesis and an alternative hypothesis. We plug our data in and we get a P value as output. We check whether the P value is below our significant threshold. If it is, then we reject the null hypothesis. So really, we don’t have to understand the augmented Dickie Fuller test. We just have to know what it is for. We have to know what the null hypothesis is and what the alternative is. Once we know these things, we can use the test. So what is the null hypothesis? The null hypothesis is that the time series is non stationary. The alternative hypothesis is that the TIME series is stationary. So if we find a P value less than, say, five percent, then we will reject the null hypothesis and we will say that the TIME series is stationary. And just to bring this back to Arima, how would we use this? Well, recall that this all has to do with the AI component of the Arima model. We want to difference our Time series until it becomes stationary so the process will go something like this. First, we just have our Raw Time series. Maybe it looks pretty stationary already. If so, we’ll do an ATF test. If we get a P value below the significance threshold, we’ll say it’s stationary and so we’ll then fit the rest of the A model or equivalently Woolfenden Arima, where we say that D is equal to zero otherwise will difference the data set. Then we’ll run our ATF test again. Again, if we get a P value below the significance threshold, then we’ll say it’s stationary. Since we differenced once we’ll say that D is equal to one. If the ATF test still does not reject the null, we might difference again. All right, so let’s move on to the second optional part of this lecture where we discuss stationary more in depth. So what is stationary? Previously, we’ve only discussed this concept informally, but now we are ready to be more exact. In fact, there are two kinds of stationary, strong and weak, strong stationary means that the distribution of the random variables in your stochastic process does not change over time. As a rough example of this. Suppose I take an arbitrary window over some time series, then a suppose I move this window over tão time steps strong stationary would say that the distribution over these random variables is the same no matter what tau is. In other words, no matter where I look in the Time series, I see the same distribution. There is a very formal definition for a strong sense stationary, but this is definitely not necessary to understand for this cause. In fact, in the practical application of Time series analysis, strong sense stationary is not used very often. A more practical kind of stationary is weak and stationary, weak stationary looks at first and second order statistics rather than the full distribution. As you know, first order statistics usually corresponds to the ME second order. Statistics corresponds to things like variance and covariance. You already know that informal definition of weak stationary. It’s that the mean in the covariance don’t change over time. All right. But now we’re going to look at this in a more exact way. Luckily, I think these are pretty straightforward. If you don’t find that to be the case, it’s not absolutely necessary to understand what we’re going to do next. So for the mean, it just says that the mean time T is equal to the meantime T plus tau for all tau. That makes sense. It means that no matter where we look, the mean is always the same. For the second order statistics, it gets a little tricky. It says that the auto covariance for some random variable Y at time T1 and some other random variable Y at times too is only a function of the time difference between a T one and two. Now, that’s probably confusing, so let’s think about what that means. First of all, what is auto covariance? Well, auto means self. We’ve seen this plenty of times, auto regressive model, auto encoder and so on. The auto covariance between Whyatt T1 and Wyatts two is really just the covariance between a Wyatts one. And why it’s to the auto part really just means that why it’s one and why a t to come from the same time series. Again, this is just the covariance and what is covariance? Well, we’ve learned that it is the unscathed correlation. Therefore it tells us how related to random variables are. If they are completely unrelated, then the correlation and hence the covariance will be zero. If they are related, that is, they move together either in the same direction or the opposite direction, then this value will be non-zero. If they move in the same direction, then the covariance will be greater than zero. If they move in opposite directions, then this value will be less than zero. So why does stationary mean that the covariance can be written as just the time difference between a T one and two? Well, the intuition is this T one minus two is just the distance between the two time points. That means if I pick any two time points in the series, as long as this time difference is the same, the covariance between these two random variables is the same. For example, the covariance between a time one and time three is the same as the covariance between a time three and time five. And that’s the same as the covariance between time 10 a.m. 12. The distance between all of these is two. In other words, the relationship between each value and the time series remains constant over time. This actually makes a lot of sense in terms of auto regressive models. If this relationship were to change over time, then we wouldn’t be able to fit any such model. That’s why we want stationary when we fit these kinds of models. For example, suppose our auto regressive model is way of t equal to zero point five times Y of T minus one. But imagine if this were only true for T equals two and not T equals three and so forth. Then this equation doesn’t work. In order for this equation to work, this relationship has to hold for all times. Not that this also implies that the variance remains constant over time. If it’s true that the auto covariance depends only on the time difference, then it doesn’t matter what time we pick K1. One is just equal to zero zero. Katsu two is just equal to zero zero. So that’s why if we see the variance change over time, as we do with volatility clustering, then we take that as evidence that the Time series is nine stationary. You may recall that earlier in the course we assume that stock returns were stationary, recall that when we were calculating things like the mean and variance of our stock returns, we used data from the time series of stock returns to calculate the sample mean and sample variance using our definition of stationary. Now we know that this is possible only if the TIME series is stationary. If, for example, there was a trend then trying to calculate the mean by adding up all the values and dividing by NP would not make any sense. This would imply that muite time t is not equal to more time t plus tau for all tau violating stationary. So understanding stationary not only helps us do what we are doing now with the rhema, but it also helps us understand better what we were doing before.

2. Efficient Market Hypothesis

All right, so this is the lecture where I get to destroy your dreams. I think it’s pretty obvious and also pretty understandable that most people who encounter machine learning for the first time think they’re going to use it to forecast stock prices. I mean, this is the dream when it comes to finance. But after the previous section, you know that there’s a lot to finance that you probably didn’t even think was relevant. If I were to ask you, what do you think goes into a financial engineering course, this section on Time series analysis would probably be the main focus, you’d think. Well, obviously, we start with Time series analysis going from simple linear models to deep neural networks. And we’re going to show that as we add model complexity, the accuracy of our predictions will get better and better. And furthermore, you’d think that this is all you need to know to do well in finance. If you could predict the future, then nothing about the distributions of returns even matters. You just pick the stock with the highest predicted future return and you’re done. There’s no need to discuss covariance, correlation, keratosis or distributions. That is the beginner dream. And in this lecture, I hope to wake you up from this dream. This lecture is all about the efficient market hypothesis or the image in a single sentence, the efficient market hypothesis says that you can’t beat the market. The reason you can’t beat the market is because stock prices in the market always reflect all currently available information. Because of this, stocks always trade at their fair value and there is no opportunity to purchase stocks which are undervalued or sell stocks which are overvalued. The implications of this are that you cannot outperform the market and you cannot time the market. That is, you might try to buy now because you think the stock is low or maybe you sell tomorrow because you think the stock is going to peak tomorrow. The efficient market hypothesis says that you cannot do this consistently. Sure, you might get lucky once in a while and you might think, aha, the EMH is wrong. I just made a 20 percent return. But the key is you must be able to do so consistently. The EMH says that you can’t. So that’s just the summary, but let’s dig deeper into the details of the efficient market hypothesis. Firstly, the efficient market hypothesis make some assumptions. I think these are a pretty reasonable starting point. We assume that there exists a large number of investors who are all interested in making money in the stock market. They obtain and integrate information rapidly as it becomes available. This makes sense. All major financial companies have very high tech equipment and they are on the bleeding edge of machine learning, big data processing and so forth. Once these investors integrate new information, they will make trades such that the price of any stock will always reflect that new information. Again, these seem to be pretty reasonable assumptions. Now, we’ve been talking about information pretty abstractly, what is this so-called information that investors are acting on? In fact, this is pretty broad, which is actually why investing is such a large scale engineering problem. We’ve discussed this partially in the previous section. Earlier, we looked at technical data. That’s price and volume information. We also know that there’s fundamentals data, such as a company’s quarterly reports. In addition to these two, there’s also exogenous data. This is a very broad term and it could mean any public data that doesn’t fit into the first two categories. For example, you could look at Twitter feeds or at the prices of other stocks. You might also look at the news as an example of that. If something happens in China that might affect Apple’s stock price since Apple’s factories are in China. Finally, we have insider information, and this information is not public, this might be something like you’re on the self-driving team and you just achieved a level of A.I. that allows for fully autonomous driving in the self-driving industry. We refer to this as level five. So if you know your team has achieved level five self driving ability, you might want to go and purchase a bunch of your company’s stock before they start selling their latest cars. Under certain circumstances, the use of insider information might be illegal. From these kinds of information, we can now discuss the three forms of the efficient market hypothesis, these are the week form the semi’s strong form and the strong form. All right. So what do these mean? The weak form of the e-mail is this. It says that future prices cannot be predicted using historical prices. Another way of stating this is that future performance has no relationship with past performance. Yet another way of stating this is that using technical analysis to predict future returns is not possible. Well, if this is true, then it definitely throws a wrench into our plans for this section, since it basically means you’re not going to use these methods to predict the future. However, this is weak because it doesn’t say other kinds of data, such as fundamentals. Data is not predictive. The semi strong form of the image is this. It says that stock prices instantly adjust to all forms of public information. That is, even if you have all the technical data and all the fundamentals data in the world, you still cannot beat the market. So if technical analysis won’t give us an advantage and fundamental analysis also won’t give us an advantage, then all that’s left is private insider information. The strong form of the efficient market hypothesis is that even with all public and private data, you still cannot gain an advantage over the market. In other words, stock prices already incorporate all public and private data. Now, at this point, the obvious question is, are any of the forms of the efficient market hypothesis actually correct? Let’s note that the strong form of the image is the least likely to be correct. In fact, that’s probably why insider trading is illegal. In fact, we have seen examples of people who used insider information to make large trades. As you go from strong to semi strong to weak, these forms of the EMH become more and more likely to be correct, since they each use less and less data. It’s helpful to start by taking an outsider’s view of this question that is don’t concern yourself with the details of finance at all, but just look at how the topic is discussed. Usually when people talk about the efficient market hypothesis, they talk about it in terms of beliefs. They say, I believe this is true, or they say researchers dispute if this were a settled issue, people wouldn’t use words like believe and they wouldn’t have anything to dispute. They would simply say it’s true or not true. Furthermore, recognize that this is a concept that people still dispute and discuss. If this were a settled issue, there would be no further discussion. What this should tell you is that even if the efficient market hypothesis is not true, there is some amount of truth to it. What I should tell you is that stock prices and stock returns are not easily and not obviously predictable. If all you had to do was plug in your data into an LSM and call it a day, then these discussions wouldn’t exist. But these discussions do exist because predicting the future is hard. Even if the market isn’t perfectly efficient, it’s at least pretty good at seeming that way a lot of the time. Going back to finance, what researchers do is they use statistical testing to check whether the different forms of EMH are true. This goes back to the kind of hypothesis testing we discussed in the previous section. These kinds of tests are outside the scope of this course, but the idea of testing in general should give you some idea. What you want to do is devise a test and calculate some test statistic. If the efficient market hypothesis is not true, then this test statistic should be very improbable and you would reject the efficient market hypothesis if that’s your null hypothesis. Note that such tests have been done in the past, which do reject certain aspects of the efficient market hypothesis. Also, consider the fact that there are some very famous examples of those who seemingly defy the efficient market hypothesis. Warren Buffett is one such example and he himself disputes the efficient market hypothesis. Behavioral economists are an entire field of people who don’t believe in the efficient market hypothesis. They believe there is a human element. And so actors in the market are not perfectly rational. So these seem like pretty plausible reasons why the efficient market hypothesis might not be true. It’s important to note that the efficient market hypothesis does not say that nobody will ever outperform the market. Remember that randomness is inherent in stock prices. Stock returns are random variables. And so just as it’s possible to have outliers who win big, it’s also possible to have outliers that lose big. You just never hear about them. But these are simply seen as people who are lucky or unlucky. In effect, it turns stock trading into gambling. So ultimately, why are we discussing the efficient market hypothesis in a section about Time series analysis? Well, if you’re from a machine learning background, you’re probably used to plugging in your data into a classifier and getting an accuracy of about 90 or 99 percent. If that’s what you’re expecting to see in finance, you will be very surprised in finance. These techniques are not so much about making a perfect prediction of the future, but rather, if possible, making a prediction that’s at least a little better than random. And failing that, it’s not about prediction at all, but rather about modeling. If you’re from a machine learning background, you’re probably very accustomed to carrying only about predictive accuracy. However, the modeling aspect is often forgotten. Modeling is important not just for making predictions, but for telling you what you know and don’t know. It can tell you what’s possible to know and what’s impossible to know. If you know that you can’t know something, then you won’t waste your time trying to predict it and you will understand why. So this is a much more in-depth and nuanced view of machine learning. In fact, very shortly we’ll be looking at another hypothesis called the random walk hypothesis. The random walk is a model where you actually can’t predict the future return of a stock and you can only discuss statistical properties.

20. Stationarity Code

In this lecture, we are going to be looking at stationary in code specifically will be applying the augmented Dicky Fuller test to a variety of data sets and will learn whether or not it actually fits our intuition about what is stationary and what is not. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so let’s start by importing pendas, numpty, matplotlib and the functionality fuller from stat’s models. Next, we’re going to download our airline passengers, Data said. Next, we’re going to load in our data set using pedigreed CSFI. Next, we’re going to plot our data said. So this will help remind you that this data set is not stationary, there’s both a trend and a seasonal component to this data. Next, we’re going to test out the 80 full or function UNDEF passengers. OK, so as you can see, we get a big table of numbers. What do these numbers mean? Well, we can check the documentation for stats models. The two key pieces of information we are interested in are the test statistic and the P-value. Most of the p value, as you know, the test statistic is used to compute the P value. Note that these are the first to return values in the tuple. So next, we’re going to write a little helper function called ATF, this will help us more easily check the results of the ATF test rather than trying to interpret a couple of numbers. This function takes in a series called X. Inside the function, we run the fuller function on X. Next, we print the test statistic and the P value. As is typical, we’ll use a significance threshold of five percent if the P value is significant. We’ll print out stationery. Otherwise we’ll print out non stationary. OK, so let’s test their function on the DF passengers column once again. As you can see, the p value is quite large and we do not reject the null hypothesis. In other words, we conclude that this Time series is non stationary. Hopefully this is the result you anticipated. Next, let’s test whether or not this function works for an actual stationary signal. So this will be ID noise sampled from the standard normal note that the signal is strong and stationary because the entire distribution remains constant over time. All right, and we get a very small p value on the order of ten to the minus 18, therefore we conclude that the signal is stationary as expected. Now, everyone knows about the normal distribution, but what if we try a more exotic distribution like the Gamma? Again, A theoretically we know that the signal is strong, sent stationary because the distribution does not change over time and each sample is Eid. OK, and we get another very small p value on the order of ten to the minus 16. Again, we conclude that this signal is stationary. And by the way, if you don’t know what the gamma distribution is, I’d recommend plotting this Time series and also plotting a histogram of these samples. The gamma is a very important distribution, especially in Bayesian machine learning. Next, we’re going to take the log of the passengers call, we’ll be making use of this over the next few experiments. So next, let’s run a test on the log passengers column. Not surprisingly, the result is that it’s still non stationary. Next, let’s difference the passengers call them. Remember that this is what we would like to fit in a rhema, too, so if we plot the this column, we see that the variance of the signal appears to increase over time. We can therefore guess that the signal is nine stationary. All right, so let’s run the ATF test on the first difference of the passengers, call them, note that we have to drop in because whenever we take the first difference, we always have one and a value in the first row. OK, so, again, we end up saying that the signal is non stationary. But notice how close the P value is to five percent. If you had set your significant threshold to a higher number, like 10 percent, you would have concluded that the above signal is stationary. OK, so next, we’re going to take the first difference of the log passengers will store this in a column called Dif Log. If we plot the flag, we notice that unlike the first difference of the raw passengers, the variance here does not increase over time. Perhaps then this signal is more stationary than just the Rothfuss difference, of course. We still have to run our test. So next, we run the ATF test on the deflate column again, recall that we have to drop the NRA values. All right, so surprisingly, we get a higher p value than we did when the variance was increasing. OK, so this is a good opportunity to remember that we care about stock prices in this cause, so let’s download the SP 500 a CSV. Next, we’re going to load in the S&P 500 at CCV using Peaty that reads, CSFI will assign this data frame to a variable called stocks. Next, we do a stock stat head just to remind you of what’s inside this data frame. As you can see, the final column is the name column, which can be used to filter out any particular stock ticker. Next, we’re going to grab all the rows where the name is equal to Googs and we’re going to grab the close column. Next, we’re going to take the log of the close, call him to get the log price. Next, we’re going to take the first difference of the log price column to get the log return. Next, we’re going to plot the log price as a time series. As you can see, there is a clear trend. Next, we’re going to plot the log return. So this looks pretty stationary, although the variance does seem to increase in some places. OK, so let’s try the ATF test on the log price. As expected, we conclude that the log price is not stationary. Next, let’s run the ATF test on the log returns. So the log returns are so stationary that the P-value just gets rounded down to zero. Now, just for good measure, let’s test a different stock, let’s use our other favorite stock, Starbucks, again, the same code to calculate the log price and the log return. Next, we plot the log price. Again, we see a pretty clear trend. Next, we plot the log return again, it looks pretty stationary, except that the variance seems to not be constant. So let’s run the ATF test on the Starbucks log price. As expected, it’s nine stationary. Next, let’s run the ATF test on the Starbucks log return. All right, so just like the Google log returns, the p value is so small that it gets rounded down to zero. Therefore, we conclude that these log returns are stationary.

21. ACF (Autocorrelation Function)

In this lecture, we are going to continue our discussion on how to choose a rhema hyper parameters, this lecture will focus on the key parameter, which is for the moving average. It’s kind of interesting, although entirely inconsequential, that we are discussing these backwards in terms of the order we introduced each component of the Arima model. It turns out that in the Arima model, it makes the most sense to discuss the auto regressive part first, then the moving average and then the integrated part. However, in terms of hyper parameters, it’s easiest to understand stationary first, then the Akef, which is the topic of this lecture, and then the pickoff, which will be the topic of the next lecture. In any case, let’s continue. So this lecture is all about the akef, which it turns out will help us determine the key parameter in the Yarema model. So what is the ATF? Akef stands for auto correlation function. Note that this is also known as the telegram. But before we talk about the auto correlation function, we have to first understand what the auto correlation is to begin with. Like the covariance and auto covariance, the auto correlation is similarly defined. You will understand this if you didn’t skip the second part of the stationary lecture, in case you did, let’s recap. As you know, the covariance is defined as this expected value between any two random variables. The auto covariance is simply the covariance between two specific random variables selected from two possibly different time points in a time series. That is, the auto part simply means that the two random variables came from the same time series. Otherwise, it is still just a covariance auto means self. So you can hopefully understand the semantics behind this terminology. So as mentioned, auto correlation as similarly defined recall that correlation is nice because it’s a scaled version of the covariance. As such, since we can always expect the correlation to be between minus one and plus one, we can expect the same of the auto correlation. Note that in some fields outside of time series analyses such as engineering, the terms auto covariance and autocorrelation might be used interchangeably and the scale diversion would be specified as the auto correlation coefficient. Now, these equations are nice, but how will they help us choose? Q Now that we understand what the autocorrelation is, we can discuss the auto correlation function. Suppose that we have some time series, why one up to why Big T if we take the correlation between each point in the Time series and each point in the Time series, we will get a big matrix of size that big T by Big T. In fact, you’ve already seen how we can do this, but actually this won’t even work because we don’t even have multiple samples of the Time series. We are only given a single time series. That is, we only have one version of one, one version of Y two and so on. In order to take the sample autocorrelation or auto covariance, we would need multiple Y ones and multiple Y twos. So we can’t use the typical formula for the sample autocorrelation that you saw before. In fact, it turns out that the auto correlation function is defined as if the time series were stationary, as you recall, stationary either weeks and stationary or strong sense stationary implies that the auto correlation remains constant over time. Since that is the case, the auto correlation is only a function of the distance between any two data points. Why it’s one and why T two will denote this distance tau. So T one minus two is equal to tau. Therefore the auto correlation is a function of tau. And since we assume that the auto correlation remains constant over time, we can use all of the samples from a single time series to compute the auto correlation for any tau. I understand that describing this in words can be confusing. However, the formula helps to provide lots of insight. The components of this formula should all seem familiar. First, we divide by T minus tau because that’s how many samples we are adding up. We divide by Sigma squared, since without this we would just have the auto covariance formula. And the auto correlation is the auto covariance divided by the Sigma’s of the two random variables inconsideration. But again, since we are assuming stationary, that means that every point in the Time series has the same sigma. And so that’s why we divide by Sigma twice. Finally, inside the summation we have y a t minus mu times y a T plus tau minus mu, which is exactly what goes inside the expected value for the auto correlation. In other words, after considering every component of this equation, you should be convinced that this does in fact compute the sample autocorrelation. So what is the consequence of having an auto correlation function, which is only a function of the lag tau note that we call this the lag because it’s as if we are comparing two of the same time series, except that one of the time series is lagging behind by the duration. Well, what we get is a graph. This graph plots row the auto correlation on the vertical axis and how the leg on the horizontal axis, usually such a plot is called the AKF and in some languages such as R, this function is built right in in Python. This function is included in statistical libraries such as Sipi and stat’s models. Generally you will see a plot like the following where the akef like zero is equal to one, and then all the rest of the values are smaller than one. The reason why the HCF at like zero is one is because this is just the variance divided by the variance or sigma squared over sigma squared, which must be one. If you want, you can check the equation for the sample autocorrelation to confirm this. The interesting part of the fly is that it also provides confidence intervals, these confidence intervals can be thought of as a kind of threshold. If we see any lagged auto correlations that are greater than the threshold, we would reject that they are equal to zero and hence say that they are non-zero to state it more casually. Any point we find in the Akef plot that goes outside of the confidence interval, we can consider it to be non-zero. Now it’s important to also remember the frequent test interpretation of confidence intervals, since this is a 95 percent confidence interval. That means five percent of the time one of the values in the Akef plot will just pop out of the confidence interval randomly. Therefore, if you have, say, 40 lag’s, you might expect two of them to just be randomly outside the confidence interval. For example, you might see one at like 25 and one to like 39. Usually you can use your intuition to determine that this is happening by chance and is not actually significant. OK, so now that we understand what the Akef plot is and how to look at one, how does this help us determine the order cue for the moving average process? Well, it turns out to have a very simple interpretation, a moving average process of order. Q Can be shown to have non-zero autocorrelation up to Lagu. That is, if I look at an active plot and I see that at lag to the HCF goes outside of the confidence interval, but after that it does not, then I would choose Q equals two. Usually it will be the case that the plot goes outside the confidence interval for smaller legs as well. So as another example, if I look at an active plot and I see that at like five, the plot goes outside of the confidence interval but stays inside for values greater than five, then I would choose Q equals five and the same logic applies for a Q equals 3Q equals four or any other Q. Now, you might be wondering why does it work this way and how do we know that we can do this? So I’m not going to derive it right at this moment, but I may in the future for some in-depth lectures. Well, what you can do is you can actually calculate, given a moving average process, what the theoretical autocorrelation would be. So recall that in May one process looks like this. It’s as y a time T is equal to a constant plus theta one, a coefficient times the previous error term epsilon T minus one plus the current error term Epsilon T. In this case we can calculate row one to be theta one divided by one plus theta one squared row to row three and so on are all zero. If we have an inmate to process, we can do the same thing, we find some expression for row one and row two and we find that row three, four and so on are all zero. And again, these are just statistics calculations. Therefore, it is theoretically justified, therefore, and make you process all of the legs up to queue will have a non-zero autocorrelation. And that is what we are looking for when we look at an akef PLI.

22. PACF (Partial Autocorrelation Funtion)

In this lecture, we are going to discuss how to choose the final hyper perimeter of the Arima model, which is peh the order of the A.R. component. This is accomplished by looking at the pickoff, which stands for partial autocorrelation function. In my opinion, the partial autocorrelation function is a bit more difficult to understand than the AKF. However, the way that we apply it is the same as how we apply the akef for the moving average model. So in this lecture, first we will just simply discuss how to use the F, then we will discuss a tiny bit more about what the pickoff actually is. In case you are interested, you should not feel obligated to understand the second part in order to move on to the next lecture. So what is the perceive from a practical perspective, from a practical perspective, it’s just a plot that looks very similar to the Akef and that it’s some kind of auto correlation function. The vertical axis is still the value of the function and the horizontal axis still represents some like the way that we use it is exactly the same. As for the moving average, there will be some confidence interval on the Akef plot. If we see any statistically significant non-zero legs up to something like P, this would indicate that we have an auto regressive component of order. P, for example, if the highest non-zero value is five, then we would choose P equals five as before. It’s usually the case that the values below five will also be non-zero. All right, so now that you know how the stuff works, what is the really we know that it results in a graph that looks like the AKF, but it is clearly not the same since it has a different name and a different purpose. There are two ways of thinking about the picture. The first way is to simply look at what it is in terms of its definition. The second way to look at the Sieff is to consider how it’s calculated. Looking at both of these perspectives should help you understand the. So what is the partial autocorrelation function, the partial autocorrelation function at Leitao is defined as the correlation between a why a time T and Y a time T plus tau conditioned on all the ways in between a T plus tau. So that’s why a T plus one, why a T plus two all the way up to Y of T plus tau minus one. So one way to think about this is it’s like a conditional auto correlation, whereas the regular auto correlation is unconditional. In other words, not conditioned on anything. The second way to think of the pickoff is how we calculate it, we usually define the coefficients using the Greek letter PHY and we use double indices, so we’ll have five zero zero five one one and so on. In general, these will be represented by Tau Tau. Note that five zero zero will just be one as usual. This is because this is the ECF between Y a time T and Y a time T for any T and therefore there are no in between values and it’s just the same as the. Furthermore, the same thing happens for five one one because again there are no values in between Y a time T and Y time T plus one. Therefore this just reduces to the autocorrelation at like one. What about for any other tau in this case we need to do a few auto regressions. Specifically, suppose that y half of time T plus tau is regressed on all of the ways in between a T plus 20. Note that this is just our usual auto regressive model. Next, let y had at time t be another regression again based on all the whys in between a T plus 20. This one is weird because it’s like a backwards auto regression. We are using future values to predict a past value. So both Y had a time T plus tau and we had a time T or regressed on the same set of variables, the ones in between. Next, we have the big reveal via Tau Tau is just the correlation between why a time T plus tau minus Y hat at time T plus tau and why a time T minus Y hat at time T as defined on the previous slide. That is to say fire tau tau or the akef for like tau is the correlation between any noise after subtracting off the parts that could be predicted by the regressions. You may have to repeat this to yourself a few times, but it will make perfect sense. The partial autocorrelation function is just the correlation conditioned on the in between variables. Therefore, it’s the correlation between whatever we can’t explain using those in between variables. And of course that makes perfect sense for choosing the order p in an auto regressive model. If the pickoff is non-zero, that means there is a significant correlation that cannot be explained by the in between variables and therefore we should include this correlation in our final auto regressive model. So just to put some real numbers to this, suppose that we want to know if y one is helpful in predicting Y five. Obviously the same question applies to Y two and why six y three and Y seven and so forth. This is because we assume our signal is stationary and therefore any auto correlations only depend on the time difference. But let’s stick to Y one in my five. The in between values are y two y three and Y for the pickoff will be large if there is a large correlation between Y one and Y five that cannot be predicted by Y to Y three and Y four. Therefore, we should use one in our auto regressive model for predicting Y five. That is to say, whenever we see large values in the pickoff, it indicates that these should be included in our auto regressive model.

23. ACF and PACF in Code (pt 1)

In this lecture, we are going to check our understanding of the Akef and encode this lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So the first thing we’re going to do is import plot and a plot akef from stat’s models, these are more useful than just generic HCF functions that only give you back an array, since these will make a plot and draw the confidence bounce automatically. Next, we’re going to import numpty and matplotlib. Next, we’re going to generate IDE noise from the standard normal. Just to build a point of comparison for what we will see later in this lecture, let’s plot what this noise looks like, OK, so I’m pretty sure this is not too surprising. Next, let’s call the function of plot, we’ll be looking at auto regressive models first. And as you know, the relevant plot for that is the notice that I’m using. The subplots function to manage the size of a plot. This returns an access object, which I then pass into the plot function. All right. So what do we see? Well, as expected, most of the lagged values are very near zero. The first value is one, since that’s just the autocorrelation of each point with itself. Notice how there are multiple values in which the pickoff goes just outside the confidence bounds. However, you have to remember the definition of the confidence interval. Basically, this is allowed to happen randomly five percent of the time, even when there is no true correlation. Since these values are so close to the threshold, we can intuit that it’s probably OK to ignore them. Next, let’s create an air one process and test that again to generate this process. I’m going to create a list called X1 within initial value of zero. Then I’m going to enter a loop that goes for 1000 iterations inside this loop. I’m going to create the next value by taking zero point five times the previous value, plus some Gaussian noise with standard deviation. At zero point one, we’ll call this variable X. Next, we append X to X1. When we’re outside the loop, we can’t explain it to an umpire in the next block, we plot one. As you can see, it looks pretty stationary and there’s no real observable difference between this and ID noise. It’s not obvious that each value depends on the previous value. Next, we plot the pickoff for X1, as you can see, we have a value at like one which is far outside the confidence threshold. Therefore, if we were to look just at this plot, we would conclude that our TIME series is an A1 process. And of course, we know that this is true because that is what we just created. Next, we’re going to do another experiment again for an R-1 process. I’m not going to run through the details of this code again, since it’s very similar to last time. The main difference with this code is that the coefficient for the previous X term is now negative zero point five instead of positive zero point five. We’ll see what effect this has on the Earth if we look at the plot of the Time series. Again, it’s not at all obvious that this is any different from just plain Idy noise. All right, so next we look at the picture, this time we see that there is, again, a non-zero value at like one, but this time it’s negative corresponding to the negative coefficient of our R-1 process. Next, we’re going to generate an R2 process, so this time the Time series will depend linearly on to pass values instead of just one to initialize. Our list will set the first two values in X two to be zero. Inside the loop, we use the following equation. The coefficient for the one value is zero point five. The second coefficient for the two value is minus zero point three. And again, we have Gaussian noise with a standard deviation, a zero point one. From our Time series plot, it is again the case that we probably cannot distinguish this from idee noise, although the variants in the later parts of the Times series seems to be quite high. If we look at the pickoff, we can see that there are now two non-zero lag’s corresponding to the two coefficients of our R2 process, the first non-zero value is positive and the second at non-zero value is negative. If we were to use this plot to choose the value of P for fitting in auto regressive model, we would definitely choose to. So this confirms that this method works. Next, we’re going to generate an five process to initialize X5 or create an array of all zeroes inside the loop to generate the next value of X. We will make X depend on three different lagged values rather than all five in particular. These will be the last value at T minus one, the second class value at T minus two and the fifth less value at T minus five. The weights for these will be zero point five minus zero point three and minus zero point six respectively. Again, we’ll add Gaussian noise with mean zero and standard deviation at zero point one. Notice that when we plot the process is a time series, it definitely looks a little different from Idy noise and all of the previous plots, there seems to be more of a pattern in a more wavelike structure to the Time series. Next, we noticed something interesting with this, even though our Time series does not depend on the lag, three and like four values, the corresponding values in the pickoff are still non-zero. So although we generated the process ourselves and we know that it doesn’t depend on the three and like four values, when we fit in ERP model, we will always include those terms. So this is a sign that just because you have some in between values that don’t affect the next value in a time series, this does not mean that their values will be zero. All right. So since this lecture has been pretty long already, I will see you in the next lecture to look at the akef and moving average models.

24. ACF and PACF in Code (pt 2)

In this lecture, we are going to continue looking at the previous notebook in which we look at the archives in the archives of signals we generated ourselves to test that our methods of choosing Arima orders actually works. So since we’ve already looked at the archive, this lecture will focus on the AKF and moving average models. We’ll start by plotting the act for idee noise. Note that we are now using the function to plot Akef instead of plot pickoff. However, the syntax is the same. All right, so what do we see as expected? There is no autocorrelation between any point in the Time series and any other point. Therefore, the autocorrelation is one at like zero and zero elsewhere. Next, we’re going to generate and may one process, we’ll start by creating an array of size 1000 with samples from the normal with mean zero and standard deviation at zero point one. We’ll call this array errors. As you recall, the May one process is generated by adding the current error, plus some coefficient times the previous errors. So next, we’ll create an empty list called M1. Then we’ll enter a loop that goes for 1000 iterations, the same size as our errors array. Inside the loop, we’ll check whether the Loop iteration index is greater than or equal to one or not. If it’s not, then at zero, which is a special case for AI, greater than or equal to one, we just have the normal MS1 equation. Again, I’ve chosen the coefficient zero point five arbitrarily, so the current value in the Time series is zero point five times the previous error, plus the current error at Index I. If I is equal to zero, we have the else block. Note that when is equal to zero, there is no past value. Therefore, it’s not possible to add any previous error. Therefore, for the first iteration, X is just equal to the first error. Outside the statement, we append X to the list May one and then outside the loop, we convert our M one list into an umpire, A. In the next block, we plot our MS1 process, note that again, this kind of just looks like regular noise, it’s generally very hard to tell whether something is that may process or an error process just by looking at a stationary time series. The next step is to plot the akef for our MS1 array. All right, so as you can see, we have one non-zero Akef value at like one, which is what we expect if we were to do this in reverse order, that is, use the HCF plot to find the order of the MACU process. We would look at this plot and choose Q equals one because that’s the maximum non-zero value. Next, we’re going to generate and may to process again, this code is very similar to the previous code, except that now the current value in the Time series depends on three errors, the current error, the error at like one and the error at like two. I’ve given that error at like one, a coefficient of zero point five and the error apply to a coefficient of a minus zero point three. Same as the error to process from earlier. Note that this time I didn’t bother to account for the early values in the Time series as special cases. Recall that in some pie, if you index an array with negative values, it just loops back around the end. Since those values are random anyway, it doesn’t really affect our results in any negative way. Therefore, I’ve decided to keep the code like this for the sake of simplicity. So there are no special cases for equal zero and I equals one. Next, we our may to process as a time series, again, it just looks like regular noise. Next, we plot the akef. All right, so we see what we expect, there are non-zero values like one and two, which would mean that we would choose Q equals two. And we know that this is true because we just created an M to process. Next, we’re going to generate an May three process, the wait for like one is zero point five. They’ll wait for two is minus zero point three and the wait for like three is zero point seven. Other than that, this code is the same as before. If we look at our Time series plot, I would say that there is still no discernible difference between this and the other plots. Next, if we look at the Akef plot, we see that the largest nonzero lag is three. Note that there is one other non-zero lag at twenty five. But again, as per what we know about statistical testing, this is supposed to happen about five percent of the time. If we saw this plot and did not know that we were dealing with an M three process, we would still probably choose May three rather than May 25. We know intuitively that May 25 is probably unlikely and we would also prefer a simpler model. We also see that this value is not that far outside of the confidence bounds. So most likely we would just ignore this as a fluke. Next, we’re going to generate an MI6 process. All right, so this time I’m not going to read you the weights, but you can feel free to make note of these if you would like. When we plot the M6 process, we see that it’s still pretty much just looks like all of the other stationary time series we’ve looked at. Next, we plot the Akef for our MSCs process, as you can see, the largest nonzero lag is at like six. Notice how at like four, the value does not go outside the confidence interval. However, we would most likely not consider this in May three because of the significant non-zero values at like five and like six.

25. Auto ARIMA and SARIMAX

All right, so the main reason I wanted to discuss the Akef and the pickoff was because in a Time series analysis courses, instructors often talk about how to use the Akef and the pickoff as if you’re following a set of arbitrary rules. But they never go on to show you that these rules actually do what they say they do by comparing the Akef and the pickoff to known data by looking at the acronym for known data. We can confirm that those rules actually work. Following them thus becomes natural. It’s also important to recognize that these are standard topics and time series analysis, so it wouldn’t be right to leave them out. Even so, there are important caveats. The caveats are that even if you do follow these rules, they might not lead to the best model. Of course, we haven’t yet defined what best model actually means, but we will. The real question is why do all that work when we can get a computer to do it for us, especially when a computer will do a better job than we can? That is what this lecture is all about. So it turns out that the computers of today are fast enough that you don’t need to go through all that work, especially since you might end up with a suboptimal answer. Instead, popular time series packages usually contain a function called auto arima that automatically finds the best model for you. That is to say, it tries a bunch of different settings and returns the best settings according to some criteria on what these criteria might be will be discussed in the next lecture. In the other language, for example, you can simply call a function called Auto Arima and pass in your Time series, which is very nice. In stat’s models, there exists no such function. However, there is a package called PMed Yarema, which uses stat’s models under the hood that does implement the auto arima function. So basically we have the power of auto arima at our fingertips. We just need to install a different library, which luckily still uses Stass models under the hood. Now, the PMed Arima API is a little different from stat’s models, so that would be worth mentioning briefly assuming that we have our data as a series, the first step will be to call the auto Arima function. This allows us to pass in an argument called seasonal as well as the seasonal period. M We will get to what this means later on, but it’s basically a seasonal extension of the vanilla Arima we’ve been discussing so far. This will return to us a model object from which we can make predictions. Note that this is not a static model object, so the API is a bit different. When we want to forecast, we call the model predict function. This takes into arguments which are relevant to us, which are in periods and return content and periods. Is the forecast horizon or the number of time steps to forecast and return content should be set to true if you want the confidence bounce along with your prediction. So that’s for the out of sample data. For the sample data you want to call the function model predict in sample. This takes into arguments start and end along with some others, which we won’t worry about. Technically, you can return confidence intervals for the sample predictions as well, although that is less common than for forecasts. Unlike stats, models start and end should be specified as integers which correspond to the indices of the train set in stats. Models start and end can be integers, but they can also be daytime objects or strings corresponding to the index of the the series you passed in for training. In order to better understand what Auto Arima is doing, we’ll have to discuss seasonal Arima. This will be brief since it’s not as important to know compared to the vanilla Arima. Just understand the basic high level points and you will be fine. Again, I want to mention that seasonality is not usually observed in stock prices, but stock prices do not comprise all of what can be considered financial data. And so these concepts do show up in financial literature as they should. One example of this is that the weather or in other words, actual seasons like summer, winter and so forth, affect when and where people purchase real estate. So that’s one example of where seasonality affects finance. OK, so seasonal Arima, abbreviated as SCREAMO, gives us three new hyper parameters called Capital P, Capital D and Capital Q, which are analogous with the lowercase versions of the Eurema. So now we have six hyper parameters in total. We usually write this as Arima PD to cross capital P Capital, the capital Q Subscript M where M is the seasonal period. The reason why we add the cross is because it turns out when you write the model using operators, you end up multiplying the non seasonal parts by the seasonal parts to get your full model. This is outside the scope of this course, since the notation is very messy and doesn’t really add any benefit on top of what we are already doing. However, if you want some simple intuition, we can discuss what the seasonal part might look like in isolation. So let’s suppose we want to consider only the seasonal part of the seasonal Arima, if we have capital, the one that means we difference YFC once using the seasonal period. Our notation for this is Delta sub M y of T, which is equal to Y of T, minus Y of T minus M. So this is just like regular differences, except that regular different saying subtracts from one timestep behind and that measures the slope of the trend. With the seasonal part of seasonal Arima, we subtract from one period ago, so if we have monthly data and T is equal to March, then we subtract from the previous March. Note that when we discussed stationery earlier, we discussed certain characteristics of Time series that indicate that it is not stationary examples of this where trend and changing variance. If either the level or the variance changes over time, then the series is not stationary. Seasonality is the other key component of this stationary time series should also not exhibit seasonality. So if you find a time series that does exhibit seasonality, then you would say is not stationary. Interestingly, what we found in our previous, quote, examples was that it is possible for a non seasonal Arima to model the airline passengers data set quite well, even though it has no seasonal component. In fact, it’s possible for a purely auto regressive H2 model to perfectly represent a sine wave. So it’s not that Arima cannot model seasonality, but including seasonality explicitly by using SCREAMO can lead to a better model. So without loss of generality, let’s suppose that we have not difference so that we can keep using the variable y t or you can pretend y t now represents the new difference, the time series doesn’t matter. The seasonal auto regressive part is, again, what you might expect. Y t is regressed on past data points in the Time series at multiples of M away from T, so for example, by of T depends on Y of T minus M Y of T minus two M and so on. For the moving average part, we again have the same pattern. Y of T depends on past errors at multiples of M. So that would be Epsilon at T minus M Epsilon at T, minus two M and so on. Finally, once we have this season apart, we multiply it with the non seasonal part, and that gives us the false Arima model. In code, there is actually one extra component which gives us the sorry max model, the X part is actually pretty nice and I think it will make you feel like this model can actually be applied to so-called real world data. The ex parte refers to exogenous variables. So imagine that you have a time series of length Big T at each point in the Time series, you have some feature vector of exogenous data, such as maybe the sentiment of tweets by Elon Musk or the sentiment from various financial news sites. You can include this in your model by passing in an array of feature vectors of size T bidi into the auto arima function. One caveat to this is that when you want to make predictions, you have to know what these features are. So you want to consider whether or not this is actually practical for you. In any case, we won’t be making use of this, but we will see on our output. Sorry, Max. So this is just in case you wanted to know what that means. Otherwise, consider this topic outside the scope of this cause.

26. Model Selection, AIC and BIC

In this lecture, we are going to circle back to the earlier part of the previous lecture, which is how do we define the best model, we know that Otto Arima will find the best model out of some set of models. But of course, in order to make any sense of that, we need to know what it means for a model to be the best, because these models come from the statistics literature. If you come from a machine learning background, you will recognize that machine learning people might approach this topic differently. However, Arima is a statistical method. And so generally, when we look at how these libraries work and what they do, they will be using techniques founded on traditional statistics rather than machine learning. What is somewhat interesting about Otto Arima is that we end up coming full circle. As you may recall, the reason that we use methods such as the Akef and the pickoff is because we want a more direct way of choosing hyper parameters than simple trial and error in comparison. Trial and error seems like kind of a very naive approach. And yet with Otto Arima, that is exactly what we do. It turns out that the method of manually looking at the HCF in the Sieff does not always lead to the optimal answer. Otto Arima does a more exhaustive search and therefore has a better opportunity of finding the best answer. Now, one method you may have thought of is grid search that is searching through every possible set of hyper parameters in a grid. For example, if you had it to hyper parameters to search for, this would be a two dimensional grid. For example, you might want to try different combinations of P and Q from one to 10. So it’s one one, one, two, one three and so on. If you had three type of parameters to search for, this would be a three dimensional grid. A grid search would be simply to look through every possible position on the grid to find the best combination of high parameters. Now, although Auto Arima gives you the option to do this, this is not the default behavior. Instead, Auto Arima uses a stepwise algorithm to more intelligently find the best set of hyper parameters. It’s true that computers are so fast nowadays that they allow a function like auto arima to be possible. However, a full grid search can still be quite slow. Therefore, we typically use the stepwise algorithm as the default. All right, so when we run out Arima, it’s going to use some criteria to evaluate each model. The model that gives us the best value will be the model chosen to sort of equally good evaluation criteria or the AIC and the Bisi AIC stands for a information criterion and Bisi stands for Bayesian information criteria on the intuition behind both of these is the same. When we are building machine learning models, we often have to make a trade off. This trade off happens between a model complexity and model accuracy. For a model, complexity means increasing the values of P and Q Recall that P is the number of past data points to include in the model, and she was the number of past errors to include in the model. You can imagine that as we add more and more terms, the model will get more and more accurate. In fact, if you studied linear regression with me in the past, then you know that even adding completely random noise will increase the accuracy of your model. This is not good. How do we know when enough is enough? How do we know when we’ve overdone it? In statistics, the answer is to penalize the moral complexity again, if you’ve studied with me in the past, then you know what I’m about to discuss. If you haven’t. That’s OK. But feel free to ask me about this on the Q&A if you want to learn more. It turns out that the laws function when we optimize these Arima models is the negative log likelihood. It also turns out that for the most part, minimizing the negative likelihood is equivalent to minimizing the squared error. So this doesn’t contradict anything I said earlier about minimizing the squared error of the predictions, the log likelihood is more general, however, since it can account for variance. Now, if we only look at the log likelihood, we might end up overfitting. So what we do is we add a penalty term to the negative likelihood. The main difference between the AIC and the Bisi is that this penalty term is computed differently. So just in case you’re curious, the AIC and the BSI are defined as follows for both of these, you will have two times the negative likelihood. So in these equations, l represents the likelihood. And so each of these contains the term minus to log out for the AIC. The log likelihood is penalised by adding two times the number of parameters in the model for the Bisi. It’s penalized by adding the number of parameters in the model times, the log of the number of data points. So they both do the same thing just slightly differently. Auto Arima happens to use the EIC by default, although it’s often said that both of these usually lead to the same answer anyway. For some reason, statisticians always discuss both of these simultaneously, even though you always end up having to choose just one. All right, so now that we’ve discussed the statistics way of doing model selection, I want to briefly discuss how a machine learning person might go about this task. Note that this is entirely theoretical at this point. We are not going to use this method, and I only mention this out of interest. In machine learning, we often don’t care that much about the number of parameters in a model. One reason why this is, is that you might be comparing different kinds of models. If you’re comparing a decision tree to a neural network, for example, they are not really comparable in that way. Another reason is for modern methods such as deep learning. It often doesn’t matter in the pre deep learning era. That is, before people even came up with the phrase deep learning. You will see a lot of papers on neural networks talking about this idea of comparing the number of parameters to the number of samples. This makes a lot of sense when you look at linear regression because the actual data Matrix X has a number of rows equal to the number of samples and number of columns equal to the number of parameters. So this makes sense when you consider linear algebra and the solution for linear regression and so on. People used to extend this thinking to neural networks, but today we have found that neural networks don’t actually behave badly when you have many more parameters than samples today, you can have neural networks with billions of parameters that cost the equivalent of millions of dollars to train. So it’s not really model complexity that we care about in machine learning. In machine learning, what we really care about is the ability to generalise, that is to say we don’t want our model to be accurate only on the data that it was trained on. We want it to be accurate for data that it hasn’t seen yet. This is important for pretty much everyone that does machine learning. For example, if you’re building a recommender system, those recommendations will be going to people who have not yet seen the movies or purchased the products that you were going to recommend. If you’re building a fraud detection system, you want to detect future fraud, not pass fraud, which you already know is fraud. If you’re building a time series forecasting model, you want the forecast to be accurate, not just the data from before the forecast. For many models, this is highly correlated to model complexity, which explains why model complexity was something people care about. In the past. You would see plots such as the following weather test performance starts to degrade when the model becomes too complex. So given this information, what might we want to do instead, if our task is to choose the best model? Well, why not simply check the out of sample accuracy or the test accuracy directly? In other words, why bother checking the AIC or the Besi when we can evaluate each model according to it’s out of simple accuracy. This seems like it will more closely do what we want to do rather than adding some penalty term to the sample accuracy, which we already know is less relevant in the scenario where we are using out of sample data to choose hyper parameters. We would call this the validation set rather than the test set. You might use methods such as cross-validation to choose your hyper parameters, although that will be a topic for another lecture. So that makes sense when what you care about is good accuracy. However, one reason to use the AIC and the Bisi is because we really are trying to achieve the simplest model possible. As you recall, one of our main motivations is modeling the data and explaining how it arose rather than making predictions. If this is our motivation, then it makes sense to want the simplest model possible that can adequately explain the data that we saw in statistics. Sometimes we call this parsimony. We say that we want to find the most parsimonious model.

27. ARIMA in Code (pt 2)

In this lecture, we are going to apply auto arima to the airline passengers data center. Our main goal in this lecture will be to improve the results we got in the last lecture, which shouldn’t be very difficult since we were just guessing. That is, if Auto Roraima works as it should. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so let’s start by downloading the airline passengers data set once again. Next, let’s import Pendas Nambiar matplotlib. Next, we’re going to install the PMed Rhema Library, this is necessary because this library does not come preinstalled in Google CoLab. Next, we’re going to load in our data set using our CSFI. Next, we do a [REMOVED] head, as usual, to ensure our data looks as it should. Next, we’re going to take the log of the passengers column and saw this in the log passengers column. This should be interesting to compare, since previously we found that logging the passengers column helps the model achieve better accuracy with the same Arima order. Next, we’re going to split our data into train and test with the last 12 data points assigned to be the test. Next, we’re going to import the PMed, a library. Next, we’re going to call the auto Arima function, OK, so what are these arguments? The first argument is the data that one should be obvious. The second argument is, Trece, we’re going to set this to true so that we can see which models auto Arima tests while it tries to find the best model. The third argument is suppress warnings, which we are also going to set the true, as you know, stats models has lots of warnings. So this will make sure that those warnings are not printed out. Finally, we said seasonal equal to true, since we want to fit a full seasonal Arima model and we set em equal to 12 since we know that the seasonal period is 12. OK, so let’s run this. All right, so once this is done, we call the function model that summary. So if you’ve ever worked with are, you should be very comfortable with this kind of output. All right. So basically, this is going to tell you the orders of the best model and other information, such as the AIC, the Bisi log likelihood and the P values of various statistical tests. You can also see the value of each parameter in the model and whether or not there are statistically significant. OK, so what’s our best model, it looks like our best model is an R three for the non seasonal par and for the seasonal part, all it does is the first difference. Remember that this is the first seasonal difference. So instead of subtracting from one time step behind, it’s subtracting from 12 time steps behind. You might actually want to try that in code to see if the result looks stationary. Also note that this is a much more parsimonious model than the non seasonal Arima we found earlier, which was in a Arima 12 one zero. This model has only three dates and one intercept. OK, so next, let’s call model that predicts. As input will say, experience equals and test and return confidence equal to true so that we can plot the confidence interval. Next, we’re going to plot our forecast along with the true values, so first we’re going to create an axis object and set the figure size. Next, we’re going to call the plot function on the test data frame. We’ll give this low label data. Next, we’re going to call the plot function on the test predictions. We’ll give this the label forecast next. We’re going to call the function of Phil between which will plot the confidence balance. As with stats models, the confidence bounce are returned as a T by two array. So we row represents a timestamp in each column represents either the lower bound or the upper bound. Finally, we call X legend to plot the labels. OK, so let’s run this. All right, and we see that our forecast is pretty good. Next, I want to plot a plot of the full model prediction, just like we did before. This will include both the forecast and the sample predictions. So first, we’re going to need to call the function model, predict in sample. We’ll pass in start equals zero and end equals minus one to get the full training predictions. Note that unlike stats models, we do not need to specify that we want the predictions to be accumulated. If you have differences in your model PMed Yarema, we’ll just assume that automatically. Next, we call all the usual plot functions to make our plot. So let’s run this. OK, so this pretty much looks as expected, both the training predictions and the forecasts look pretty good. One weird part about this graph is the beginning where all we see is a flat line. Since we now have a seasonal model, it’s not possible to make predictions for the first few times that this is due to the fact that we require values from a one season behind which don’t exist. OK, so next, since we know that our model might perform better on the log data set, we’re going to run our auto arima again, but on the log passengers, otherwise, all the arguments to this function are the same as before. So let’s run this. All right, so now let’s check model that summary. So as you can see, this finds a slightly different model, then the Ottawa rhema on the non logged data set. Now we have two auto regressive coefficients and one seasonal moving average coefficient. Again, we do seasonal differences rather than non seasonal difference in. All right, so again, we have a plot of the forecast against the true values, I’m not going to explain this code since it’s exactly the same as the previous code. So qualitatively, the results still look pretty good. Next, let’s look at the full train and test predictions again, this is the same code as before. OK, so the plot is kind of hard to see, since the first few values stretch all the way down to zero, you could block this if you wanted, but I’m not going to bother. Now, since both of these models look to be performing pretty well, it might be a good idea to test them quantitatively, that is to use the IMC. You’ve seen this function before, so there’s no need to explain it again. So let’s run this. All right, so this is kind of the opposite result from what we saw previously in this case, the non diversion performs better. Interestingly, we get an army of about 18, something which is on par with our previous Arima 12 one zero model, the one we found by just guessing. Now, the next thing I would like to do in the script is to test Otto Arima itself. I want to know, can Otto Arima find the model that we found last time? So I’m going to call Otto Arima again on the log passengers. This time I’m going to set a few more parameters into this function for Samona, said Max equal to 12 and Max Q equal to two. If you look at the documentation. These values are set quite low by default. So if we want auto arima to even have a chance of finding the model that we found, we have to set them higher. Note that we also have to set a parameter called Max Order. This value has a limit on the total sum of all the Arima orders, which promotes finding a simpler model. Again, the default value is five, which is very small. So in order to have a chance of finding our previous model, we need to set this higher. I’ve said it to 14 since that’s the sum of 12 into next. We have an argument called stepwise. This controls whether we use an intelligent algorithm to search the hyper parameter space or if you use a naive grid, search the default value for this is true. So for now, we’ll leave it like this and see how it performs. Finally, we set Seasonale equal to false, since our goal right now is to compare these results to the results from the previous script. OK, so let’s run this. OK, so what’s interesting about this is that it doesn’t even try that many values. In fact, we never get past P equals five, but perhaps this is a better model than what we found before. If we do a model that summary, we can see that the presumably best model is a rhema for one two. Next, let’s plot our model forecast. OK, so we can see that this forecast is pretty bad. Next, let’s plot both the terrain and test predictions together. Again, not so great. Finally, let’s check the arms, see? So as you can see, it’s pretty high relative to what we’ve seen before. OK, so at this point what I want to do is to compare what we just did to a full grid search. So let’s go back up. And let’s set stepwise false. And let’s run this all again. OK, so this time we can see that auto arima is actually testing out all the possible combinations of models, not that it would not be a good idea to do this if you had a full screamo model that is both the Nazis in a part and the seasonal part. Again, this goes back to the concept of the curse of dimensionality. Currently, we’re only testing out two hyper parameters, which is a two dimensional grid. If you have on average 10 values of each type of parameter, then that would be one hundred values to test. If you had the seasonal part to, then you would have 10 to the power for values to test. That’s 10000, which is 100 times more than one hundred. OK, so in any case, let’s run model that summary. As you can see, the best model appears to be a rhema 12 one one. This is very close to our manually found model, which was 12 one zero. OK, so next, let’s plot the forecast. OK, so that looks pretty good. So already we can see that perhaps auto Arima might not necessarily find the best model with the default settings. Next, let’s plot the terrain and test predictions together. Again, this looks pretty good. Finally, let’s print out the pharmacy. OK, so we get about twenty three something. This is actually quite interesting because this is a worst model than the model we found on our own. As you recall, a remote 12 one zero achieves 18 something or AMZI. Furthermore, that model has less parameters than this one. So this model actually gets a higher penalty for having one more parameter. So what this tells us is that finding the best AIC is not the same as minimizing the out of sample error. They are similar ideas in concept, but one is looking for a simpler model purely to look for a simpler model, while the other doesn’t actually care about model complexity, but rather accuracy. So these two ideas are related, but they are not the same.

28. ARIMA in Code (pt 3)

In this lecture, we are going to apply everything we learned about Arima and apply that to stock prices. So this goes back to the fundamental question we asked at the beginning of this section. If we make use of time series analysis, can we predict stock prices? After all, stock prices are a time series, just like airline passengers, just like sales data and so forth. I think by the end of this section, we’ve come to learn a few basic facts. Firstly, that Arima is a pretty well accepted staple of time series forecasting. Secondly, is that a more modern approach to finding the best Arima orders would be to use auto arima rather than trying to pick the orders manually. So this lecture is all about taking everything that we learned and applying that to stock prices. As a side note, I want to mention that this lecture contains no new code, which you haven’t seen before. So if you feel like exercising what you’ve learned, you already know what we’re going to do if you want to try to do it all on your own. That would be highly encouraged. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so let’s start by downloading the S&P 500, it’s ESV. Next, let’s install PMed Yarema, unfortunately, because this library doesn’t come with Google CoLab, you’re going to need to install it manually each time. Next, we’re going to import PMed Yarema, as well as pandas, slumpy and matplotlib. Next, we’re going to use ESV to load in our data as a data frame. Next, we’re going to check the fact head to remind ourselves what the data looks like. Next, we’re going to filter out Google’s clothes prices and assign this to a variable named GOOG. Note that you also have the option of filling a log prices instead for this lecture, we’re going to work with the nonlawyers prices, but you’re encouraged to try both on your own. Next, we’re going to apply Google’s closed prices so that we know what auto Auriemma is trying to model. Next, we’re going to split our data into train and test set, we’ll use 30 days as the forecast horizon. Next, let’s call the function auto arima note that we pass in seasonal equals false sense. If we pass in a seasonal equals true, then we have to pass in the seasonal period. And of course, it’s not clear what that is. OK, so let’s run this. Next, let’s run a model summary. OK, so we see that the best model found is in a rhema three one zero. So does that make sense? The difference in order makes a lot of sense, because as you’ve seen, returns are pretty stationary. Although the first difference of the raw price is not exactly the return, we can imagine that they are related to. Be sure you might want to plot the first difference and check it yourself. The next thing we see is that the auto regressive component has order three. That kind of makes sense. It says that the last three days of data might be useful in predicting the next return. OK, so let’s look at model yet. Ms. This provides some interesting information about the model, but not much we didn’t already know the important part of this is that it gives us a nice way to retrieve the order of the model. As you recall, we need to know this, since it affects which days we can make predictions for. For example, if this is equal to one, then we can make a prediction on day one, because defensing always makes the first row and any value. OK, so next, we’re going to write a function called plot result. This function is very similar to the ones we wrote for the previous scripts. Basically, the point is to plot the data itself, the IT values for the sample data, the forecast for the out of sample data and the confidence balance. So as input into this function, we take in a trained model the full data, the train data and the test data. The full data is just the data before we split the data into train and test. The first thing we do in this function is to make use of the jet programs function we just looked at. This is a dictionary. So first we look at the key order, which gives us the order of the fitted Arima. Next, we grab the second component, which gives us back the difference in order. The next we call model predict in sample to get the train predictions. Note that the starting value is the next we call model that predict to get the out of sample predictions along with the confidence balance. Next, we call our plotting functions. First we plot the full data. This will be given the label data. Next, we plot the train predictions. This will be given the label fitted. Next, we plot the test predictions. This will be given the label forecast. Next we call the function of fill between to draw the confidence bounce. Lastly, we call a legend to show the legend. So let’s run this. And let’s test that our plot result function. So as you can see, the fitted model looks pretty good. Of course, you shouldn’t trust a plow like this because it’s very small. Unfortunately, it’s too small to see whether or not our forecast is good, which is the part we actually care about. So next, we’re going to write a function called a plot test, the point of this function is only to plot the test period. By seeing it up close, we’ll get a better idea of how well our model actually performs. So in this function, we only take in at two arguments, a fitting model and the test data frame. Inside the function we call models predict to get the predictions and the confidence bounce. Next, we plot the true test data along with the forecast and then the confidence bounce. Actually, this is the same code from earlier, so hopefully you’re comfortable with it by now. Next, we test out our new functional test. So what do we see? Well, we see that our forecast is not actually that good. Remember, this is over 30 trading days. One nice thing about this is that it does seem to capture the average quite well. In addition, the true price always seems to stay within the confidence bounds for the most part. Of course, it’s worth asking whether a model like this could actually predict what will happen over the next 30 trading days if Elon Musk goes on Twitter and starts making crazy tweets. I don’t think models like this will be able to predict that. OK, so as you know, one important question we have to ask at this point is, do these models perform better than the naive forecast? Is using these models worth it? So let’s define the Ahmedzai function, which you’ve seen before. Next, let’s print out the pharmacy for our Arima model and for the naive forecast note that for the naive forecast, the train prediction, it can simply be a single value, which is the final value of the train series. This is because series and lumpia was broadcast during subtraction, so you can subtract the scalar from an array or vice versa. All right. So what do we see? Well, we see that although we’ve done a lot of hard work to train this model, it is, in fact the naive forecast that wins. OK, so next, we’re going to do the same thing for Apple stocks. OK, so note that for Apple, there was a big dip in the price over the last period, it’s worth asking yourself, is such a drastic event predictable by NRMA model? Remember that these models tend to follow patterns if there’s a trend, it follows the trend. If they’re seasonality, it follows the seasonality. But I think intuition, it tells us that models like these do not predict anomalous events. For example, a stock like this rising upwards very quickly and then dropping off all of a sudden. That’s not really a pattern. That’s more like the opposite of a pattern. OK, so next, we’re going to split our data into train and test. Next, we’re going to call Otto Arima again, note that seasonal is false. So let’s run this. OK, so here’s the model summary. Again, the fact that the difference in order is one is not surprising, what is kind of surprising is that this model is very different from the previous model. Here we have P equals two and Q equals two. Earlier we had a peak of three and Q equals zero. Next, we’re going to call a function of plot result. So as you can see, even with this tiny plot, it’s obvious that the true data goes way outside the confidence bounds. However, I think this makes sense in terms of the model we’re using. The model seems to just want to carry on the existing trend. Philosophically, I think that’s what most models would do in the context of pattern recognition and machine learning. That makes total sense. OK, so next, let’s call the function plot test so we can see the forecast period of close up. All right, and we see that, in fact, the prediction is pretty far off. However, note that in the short term, the prediction is not that bad. Over the first few days, the true values stay within the confidence bounce for the most part. Now, of course, we still have to check the root mean squared error against the naive forecast. Again, the Niyi forecast performs better than the Arima. Next, we’re going to do the same exercise, but for the IBM stock prices, again, let’s start by plotting the data. Next, let’s split the data into train and test. Next, let’s call Auto Arima. Next, let’s check model summer. OK, so what do we see, in fact, for IBM, the best fitting model is a random walk. What does this mean? It means that IBM stock returns are completely unpredictable, given lagged values or lagged errors. Next, let’s call plot result. OK, so nothing out of the ordinary, it does look like the confidence bounce and closed the true values in the forecast. Next, let’s call playtest. Again, we see that the confidence bounce and close the true values, in fact, this prediction doesn’t look that bad, the actual price crisis through the prediction line multiple times. So what happens when we check the root mean square error against the naive forecast? Well, we see that they are the same since this is a random walk model with no drift. The forecast is just the last known value and it’s the same as the Niyi forecast. OK, so next, let’s try Starbucks again, let’s start by looking at the plot. Next, let’s split the data into train and test. Next, let’s call Otto Arima. Next, let’s check the model summary. OK, so here is another random walk, Oremus zero one zero. Note, however, that this is a random walk with Dreft since the model fits with a non-zero intercept term. Next, let’s check a plot, Resul. OK, so it’s a bit hard to see what’s going on during the forecast. Next, let’s check Plott test. All right, so this doesn’t look too bad, it’s decently accurate for the first few values and then it overestimates the price when the price drops, although most of the time the price does stay within the confidence bounds. So the lesson is, don’t ignore the negative side of your confidence balance. Those values are perfectly achievable and they result in a loss of money. So what happens when we check the root mean square error? Well, although we detected that this was a random walk with a non-zero drift, it turns out that assuming a random walk with zero drift would have been better, the naive forecast wins again.

29. ACF and PACF for Stock Returns

In this lecture, we are going to come back to the Akef in the pickoff and apply them to stock returns. So why are we doing this now in this kind of weird order? Now, you might at first think these lectures are totally disorganised. Why are the Arima coding lectures all split up? Why do we look at the HCF in the pickoff so many lectures ago only to return to them now for stock returns? Well, there is a method to the madness. I think it’s important to start with the most obvious approach. First, in the case of a Arima, the most obvious approach, if you have the choice between Otto Arima and the old school methods, I think most of you would choose Otto Auriemma. And yet we’ve seen that Otto Arima does not always give us what we might consider to be the best model. That is when we disagree with the AKG information criteria. So the goal of this lecture is to take a step back and consider what would we do if Otto Arima didn’t exist? This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so let’s start by downloading our S&P 500 at CSFI. Next, let’s import our usual library’s pandas, dumping matplotlib plot Akef and apply. Next, let’s a Naqvi using a PDF read CSFI. Next, let’s filter out all of Google’s clothes prices. Next, let’s write a helper function to calculate the log return of a series, so as input, we’re going to take in a series called Price. Next, we’ll calculate the log price by taking the log of the price. Next, we call the function to take the first difference over the log price, which is the log return. OK, so you might be wondering why are we taking the log return, why is this what we care about? Well, you may have forgotten by this point, but remember that when we apply the AMA model of Arima, we want that signal to be stationary and our stationary lectures. We already established that taking the first difference over the log prices, i.e., the log returns, yields a stationary signal. Of course, you could do the test again in this script, but I’ll leave that to you as an exercise. Next will apply a function to calculate the log return on Google stock prices. Max, let’s do a deft head to make sure our data frame contains the right thing. OK, and it does. Next, let’s plot the chief of Google’s log returns. Remember, this is for choosing the order of the auto regressive model. OK, so what do we see? Well, pretty much that there is no autocorrelation for any order greater than zero. We see a couple values just slightly outside the confidence bounds, but these are likely due to chance, as you recall, even idee noise can lead to the same plot. OK, so according to the pickoff, we would choose P equals zero. Next, let’s check out the autocorrelation or the assif. So this looks pretty much the same as the pictures, in other words, we would choose Q equals zero also. So what can we conclude if we use the scarf and ACF method to choose our Arima orders, we would choose Arima zero one zero. What is Arima zero one zero? Well, that is a random walk. OK, so maybe that was just a fluke. Let’s now try Apple’s stock price instead. All right. So this is all the same code as before. Let’s filter out Apple’s close price. Next, let’s calculate Apple’s log returns. Next, let’s look at the picks for Apple. OK, so what do we see again, pretty much the same picture. There are really no significant values and therefore we would choose P equals zero. OK, so what about the akef? Again, this looks pretty much exactly like the Earth, so we choose Q equals zero, so what do we conclude? If we go by the Axford method, we would conclude that Apple’s stock price follows a random walk. OK, let’s try again this time will do IBM. All right, so here’s the akef. And again, we see the same pattern we choose P equals zero. How about the HCF? Again, the same pattern we choose Q equals zero. Notice how closely the ATF and the ATF tracked each other for all the plots we’ve looked at, consider why this might be the case. It should make sense when you think about the formula for calculating the ATF. In any case, for IBM, it’s pretty much the same situation. We choose a rhema zero one zero or in other words, we choose random walk. OK, so let’s try this one more time, this time for Starbucks. All right, so here the pickoff. Now, notice for Starbucks, there’s kind of a non-zero value at like three, but I would consider this pretty unlikely to be actually significant. I would be comfortable saying that P is equal to zero. How about for the Akef? Again, it looks just like the picture. So let’s say Q is equal to zero. All right, so for every stock we looked at in this lecture, if we use the AKF PKF method of choosing Arima orders, we would choose a zero one zero every time. In other words, the Randhawa is actually a pretty nice, fitting, parsimonious model for stock prices.

3. Random Walk Hypothesis

In this lecture, we are going to discuss yet another hypothesis called the random walk hypothesis, which is more mathematical and more in line with the topic of the section, which is TIME series analysis. In fact, we’ll see very shortly that the random walk is just a special case of the Arima model, which we will be studying in this section. The random walk hypothesis is similar, but not the same as the efficient market hypothesis. Put simply, it says that stock prices follow a random walk. Of course, you may not know what a random mark is yet, but that’s what this lecture is about. When we did our simulation earlier, we actually assumed that stock prices follow a random walk. Although you may not have known this at the time because of the nature of random stocks. If stock prices do in fact follow a random walk, then they are unpredictable. The rest of his lecture will show you how. But first, let’s discuss some of the history behind the random walk hypothesis. Firstly, the mathematical concept of random walks has existed for a long time. As you’ll see, it’s just a matter of process. And so it’s something you would normally learn in probability class. The random walk hypothesis is specific to finance and stock prices in particular. It was popularized in the 70s when a book called A Random Walk Down Wall Street was released. In fact, this was the book that also popularized the efficient market hypothesis. Note that both the random walk hypothesis and the efficient market hypothesis lead to the same conclusion, which is that you can’t beat the market. Now, of course, there are people who don’t believe in the random hypothesis. And so another book has come out called A Non Random Walk down Wall Street. Interestingly, this book came out almost 30 years later after a random walk down Wall Street. So it’s not as if the random hypothesis and the efficient market hypothesis are ideas which are easily debunked. Later in the section, we’re actually going to fit models to stock prices and we’ll find that sometimes the best fitting model is, in fact, a random walk. So what is a random walk? Well, probably the simplest random walk works like this start at any price. Then in order to generate the next price, simply pick either plus one or minus one with equal probability. So P one is equal to zero plus one where each one it can be either minus one or plus one, then generate two from P one in the same way by picking either plus one or minus one with equal probability and then adding it to P1. Then we find P three and then we find P four and so on. So this is a random walk. Basically you can imagine yourself walking on the sidewalk in one dimension at every step you either decide to take one step to the left or one step to the right based on the result of a coin flip. Your walk is then a random walk. Notice one important property of the random walk. It’s impossible to predict the next value. You only have a 50 percent chance of getting it right. In other words, your ability to predict the result of your walk is the same as your ability to predict the result of a series of coin flips. Now, we know that changes in stock price aren’t just minus one and plus one, but can be real valued. In fact, we spent a lot of time in the previous section of this course trying to figure out what is the distribution that stock returns follow. Let’s assume for now that the noise term is Gaussian. What would our algorithm be for generating stock prices? Again, we start at p0 equal to some arbitrary value. To find the next price, we first sample E from our Gaussian, then we add a P zero plus one to find P one the next price. We do the same thing to generate P2 and P3 and so forth. This should sound familiar because it’s exactly what we did in our price simulation exercise from the previous section. In fact that was exactly a random walk. Notice again how we can’t predict P one from zero or equivalently we can’t predict P one minus P zero, which is just E one, which is Gaussian noise. We can only predict one insofar as we know its expected value. Here’s something interesting we can do that helps us understand why working with log prices is valuable, the general formula for a random walk with a drift is as follows. Muse called the drift term, and it’s considered to be constant. If you’re thinking of a time series, this would control the trend of the Time series. E of T is a Gaussian with mean zero and some variance sigma squared. In this case, time T and time T minus one are the log prices at time T and time T minus one respectively. Note that if I take part time T minus one to the left hand side, I get time T minus time T minus one, which is the log return. If we were working with nonlawyers returns, this wouldn’t be as convenient, since we would need a P of T minus one in the denominator to represent the return. What this says is that the log return is just the thing on the right hand side, which is just the Gaussian with Meenu and Variance Sigma squared. So the random walk model goes hand in hand with log prices and log returns. In fact, this model is the basis for the Black-Scholes formula which earned the Nobel Prize in economics. Now, the big question is, of course, is the random walk hypothesis correct? Well, let’s recognize that there are some hidden assumptions in the random walk model. First is that the log returns are ID independent and identically distributed. We have seen that this may not be true because we have observed volatility clustering. If the volatility changes over time, then by definition it’s not identically distributed. Furthermore, if the volatility in one period has some relationship to nearby periods, that is high. Volatility is clustered with other high volatility, then it’s also not independent. At the same time, the random walk model is convenient and easy to work with. We will find that when we fit Arima models to stock prices, sometimes the best fitting model will be a random walk. So it wouldn’t be wrong to say that sometimes for certain periods of time, stock prices do look like they follow a random walk. As with the efficient market hypothesis, it’s possible to use statistical tests to determine whether or not stock prices follow a random walk. Now, since this is a section on Time series, we’re going to do some time series analysis on random walks. Let’s recognize that a random walk is just a specific instance of a Markov chain. If you’ve ever taken any of my courses on NLP or reinforcement learning, you should be familiar with this concept. The basic idea is this. Consider the sentence. The quick brown fox jumps over the lazy dog. If I gave you the sequence, the quick brown fox jumps over the lazy, how can you predict the next word of this sentence? Well, one solution is to build a probability distribution so you have the probability of the word a time t given the word a time, T minus one, given the word of time, T minus two and so on. We call such a model a language model. Well, to get to the point, the mark of assumption says this, it says that instead of the word a time t, depending on all previous words, it only depends on the most immediate preceding word. That is P of word a time T given word, a time T minus one word at time, T minus two and so on is equal to P of word a time T given word of time, T minus one. Now you might think, OK, that’s fine, but let’s make this a little less abstract. Suppose I give you the word lazy and I ask you to predict the next word in my sentence. Of course, there are many possibilities. It could be lazy dog, but you’d probably be cheating because that’s the sentence I gave you earlier. It might be lazy programmer, who is the author of this course, but again, you’re going to use exogenous data to make your prediction. How about lazy student? In fact, it’s quite difficult to know with any certainty exactly what the next word will be, given only a single word. Consider the word. The the next word could be practically anything. So the lesson here is that the mark of assumption is an extremely strong modeling assumption. At the same time, it’s quite useful. So let’s assume we have a Gaussian random, OK, this is X 50 equals to X of T minus one plus F.T. Where it is Gaussian distributed with mean zero and variance sigma squared. In this case, we can see that X of T is completely determined by a Gaussian distribution center, that X of T minus one with a variance sigma squared. That is, it does not depend on any previous values in the series, not X a T minus two, not exit T minus three and so on. Therefore the Gaussian random walk forms a Markov chain. So if stock prices follow a Gaussian random walk, then the next obvious question is how do we forecast remember that because the next step is essentially random. The best we can do is find the expected value. Well, the expected value of a Gaussian with mean X of T minus one is just the mean X of T minus one. So what does this say? It’s saying that if your stock price follows a random walk, then your best guess for the next stock price in the series is just the previous value. We cannot do any better than this. Notice that this justifies our method of filling in missing data, which is to copy the previous stock price forward in time. Now, as you know, often when we make estimates and statistics, we also want to quantify how confident we are in those estimates. Let’s suppose we start at Ezzati and we want to forecast tão steps into the future to find X of T plus how we already know the expected value of X 50 plus tau. It’s just X of T, the same value we started with. But what does this variance? Well, we can use our price simulation formula. We know that X 50 plus one is equal to X of T plus 80 plus one. Based on that, we also know that X of two plus two is equal to accept plus one plus two plus two. We just added one to all the time indices. However, we can substitute x 50 plus one and then we would get X of T plus eight plus one plus eight plus two. And then we keep following this pattern until we get to 50 plus tau. So X of two plus tau is equal to acts of T plus eight plus one policy of T plus two all the way up to 80 plus tau. Now luckily we did something exactly like this in the previous section. If all the E’s are Gaussian with mean zero and variance sigma squared, then there’s some as mean zero and variance tau time sigma squared. Therefore we can say that the variance in our estimate increases linearly with tau. More commonly we work with the standard deviation so we can say that the standard deviation of our forecast increases with the square root of the number of forecasting steps. Let’s consider a well-known theorem from statistics, the central limit theorem. We know that our forecast, the Time T plus tau is the last known price of T plus the sum of a bunch of noise terms. Recall that the central limit theorem says that this sum tends to a Gaussian distribution. And so even if your returns do not necessarily follow a Gaussian distribution in the short term, what happens in the long term? Well, in the long term, you’re just adding up a bunch of random variables. And due to the central limit theorem, their distribution approaches a Gaussian. I want to end this lecture with a tale about a famous experiment run by The Wall Street Journal in nineteen eighty eight and this experiment called the Dart Throwing Investment Contest. Professional stock traders from the New York Stock Exchange competed against dummy investors who simply threw darts on a board to choose stocks randomly. Now, granted, one might argue that throwing darts is not actually random and there may have been better ways to make random choices. In any case, they found that professional investors beat the dummy investors sixty one out of one hundred times and the dummy investors won only 39 out of 100 times. So you might think it’s better to go with a professional investor rather than just picking stocks randomly. However, the professional investors only beat the market fifty one out of one hundred times. This is why it’s often advised not to use active investing, although your bank will tell you otherwise. Just don’t forget your bank is there to sell you things, not to give you good advice if you buy into an actively managed fund. First of all, you may only have a 50 percent chance of beating the market and on average you will match the market. However, the fees for actively managed funds are much higher than passively managed funds. Therefore, if you invest in the market itself, your fees will be much lower and you will have the same expected return anyway.

30. Forecasting

In this lecture, we are going to discuss how to forecast with a Time series model now, of course, you might say I already know how to do this. I just call models that predict. But that is not the answer because you don’t actually know what’s going on. And when you call that function, the purpose of this lecture is to actually expose a common mistake, although obviously, if you ever want to implement your own forecasting model from scratch, these ideas will be helpful. OK, so recall that one way to think of an auto regressive model is that it’s just linear regression. You can implement an auto regressive model yourself provided that you organize your data in the correct way. Let’s recall what that is. Suppose that we have a time series y one up to Y 10. Also, suppose that we want to build an AR three model. So P equals three, which means the number of columns in our data set is three. Therefore, we organize our data into tables as follows. The first row is y one way too and Y three, the corresponding target is Y for the second row is y two y three and Y for the corresponding target is Y five. All right. So you get the idea. Now, at this point in the course, we’re a little more sophisticated than we were before. We know that in addition to the training data, we want to have test data so that we can be sure that our model actually predicts the future. So let’s say we would like the last three samples to be the test data. All right. So we call the first four rows, the training data, X train and Y train. Then we call the last three rows the test data X test and we test what’s next. Then we fit our model, so we instantiate a linear regression object, then we call model Duffett X, train Y train, then we can calculate the out of sample accuracy of our model by calling model that score X test y test. We can also make future predictions using models that predict X test. Unfortunately, this is all wrong. I repeat, do not do this. So this is something I alluded to earlier in my lecture on the naive forecast in that lecture, I said that one of the bad things marketers do is they write these articles about predicting stock prices with the firms that make this huge mistake. Well, this lecture is all about what that huge mistake is. As you’ve seen for yourself after going through this section, it’s very easy to get a plot with a model. Predictions almost exactly track the true value. How can this be when stock prices are so hard to predict? Well, the reality is these models are doing nothing but making a prediction very close to the Niyi forecast. In other words, these models are basically just copying the last value. They’re not learning the underlying pattern. The reason they look so close is because these plots typically contain so many data points that you don’t get to see them up close. But how do these marketers get their place to look so good on the train set and the test set? As you know, the model should never see data from the test set and therefore should have no opportunity to copy those values. Well, in fact, that’s exactly where the mistake is if we split up our data more carefully, what we would want to do is this. Let’s suppose again that the last three data points are for the test said these are the data points that we want to forecast. So that’s why eight. Why nine and why 10? Why one up to why seven belong to the train said, well then what’s the problem with defining our X train on our X test matrices as we did before? Well, as you can see, we are making a mistake by including true test values as inputs into the model. This is not allowed. To see why this is more clearly, let’s look at how we would actually compute a forecast, suppose that today is day seven. We are trying to make a forecast for day eight, day nine and 10 since today is day seven. We can only use way one up to seven to make predictions. So, for example, if today is July seven, then obviously we can’t use the values on a July eight, July nine or July 10 because those days haven’t happened yet. OK, so that should be fine. We can calculate in the usual way we had A is equal to B plus 51 times Y seven plus five, two times Y six plus five, three times Y five. That makes sense. Next we have to calculate why. Hat nine. Now you just follow the pattern. So you might say why had nine is equal to B plus Y one at times Y eight plus Y two times Y seven plus five, three times Y six. Unfortunately, this is all wrong. Remember, we do not know why. Ed, the best thing we can do is plug in. We had a our prediction for day eight. So in actuality we had nine is equal to B plus one times we had A plus five, two times Y seven plus five, three times Y six. We can repeat the same pattern for we had 10, which will depend on why had nine and we had eight and the true value on day seven y seven. So just in case this isn’t clear, let’s summarize what’s happening and pretty much all of these blog posts and even some paired courses that I’ve seen claim to show you how to predict stock prices with Alethia jobs. First, they make the mistake of not splitting up the data properly. That is, they include test data as inputs into the forecast period. This isn’t necessarily wrong if you make it clear that you are only trying to predict one day ahead. I’ve never seen anyone make this admission to by doing this. All they’ve done is created a model that copies the previous value in a time series or very close to it. You’ve already seen in this course that this basically leads to the predictions being very close to the true Time series. As long as you’re zoomed out far enough, they’re basically indistinguishable. And three, to bring this back to the new forecast, generally, this would be OK if you make it clear that you are only trying to predict one day ahead. And do you check that your forecast is better than the new forecast? Obviously, this must be done on the out of sample data or the test set. These models will not beat the forecast. Note that this is obvious when you consider the API for stats models, as you recall, it works something like this. First, you create an Arima model. When you call the Arima constructor, you pass in the training data, then you call model Duffett, which takes in no arguments. Later, when you want to get your sample predictions, you call results predict. The arguments to this are only the start and end indices, not any data. If you want to forecast, you call results forecast. Note that the only arguments to this function is the number of forecasting steps. Therefore, there is no point at which your model ever sees any data points from the test data. Your model only sees the training data which is passed in when you instantiate the model. So what is the lesson here? The lesson is that whenever you see plots like these, pay special attention to the actual code. Are these models actually predicting stock prices or are they just finding very computationally expensive ways to copy the last value in a time series, most likely performing worse than the naive forecast? Furthermore, we know that it’s not stock prices that we care to predict, but rather stock returns. If you’ve had experience with a machine learning, then, you know, it’s probably not a good idea to use prices as inputs or outputs to your model. Machine learning models are generally not good at extrapolation. So, for example, if the prices that you trained on were between one hundred dollars and two hundred dollars, then maybe it’ll learn something for that range of prices. But what about in the future when your price goes beyond 200? Your model has never seen this before and therefore it probably won’t know what to do. So be suspicious when you see marketing material like this and don’t use test data in your forecast.

31. Time Series Analysis Section Conclusion

In this lecture, we are finally going to conclude this section and summarize everything we learn. This section was all about classical time series analysis. We covered two Staples of Time series analysis, which are the whole winters’ model and Arima. Well, winter started from a very simple concept, instead of just taking all the values in a time series and calculating the average, we looked at the moving average. This kind of thing makes sense when the data changes over time. We started with the simple moving average and extended this concept to the exponentially weighted moving average. From there, we looked at how the exponentially weighted moving average could actually be seen not just as a way of computing an average, but as a predictive model. Seeing this as a predictive model opened a new door for us, specifically allowing us to model the trend component. This gave us holds linear trend model, which improves on exponential smoothing by predicting lines at any angle rather than just horizontal lines. From there, we learned about the whole winters’ model, which adds seasonality to the whole model. The next topic we looked at was a more general model, the Arima method, Eurema method is more general because it doesn’t assume anything about the structure of your data. It’s not explicit about trend and seasonality. Like whole winters, we learn that A comes in three parts. The auto regressive part of what AP the integrated part of already and the moving average part of auto cue. The auto regressive part is based on past values of the Time series, whereas the moving average part is based on past errors of the model. The integrated part specifies how many times you used different saying in order to make your time series stationary, we looked at how to test stationary using the augmented Deqi Fuller test. Initially, we tried to guess Arima orders sort of as a first crack at finding a good set of values for PD and Q. To improve on guessing, we learn that there is a more scientific approach using the SCAF and the SCAF. However, with modern advances in computing, it’s often accepted that auto arima is an even better method of choosing Speedy. And Q This allows us to efficiently search through seasonal orders as well. For stock prices, seasonality is less relevant since stock prices typically don’t have seasonality. Of course, this isn’t set in stone and others have said that some seasonal patterns do exist in stock prices. You are certainly encouraged to test that out for yourself if you think it could be true. In the end, we discovered that 30 day forecasts, which are somewhat long term, are better predicted by the naive forecasts rather than with the rhema. This is equivalent to the random walk model, and it shows that even if the random walk model might not be exactly correct, it’s still pretty hard to beat it. Now, don’t be fooled. A lot of beginners might see this and think, well, what a waste of time you could have just said. Stocks follow a random walk and of course, beginners being beginners would be missing the point. Although the conclusion is simple, the methods and the reasoning are not. You now have a much better model of how stock prices behave from a statistical perspective. One useful trick for understanding Arima is to think about how you would implement such a model yourself. This opens a lot of doors when you really think about it, because instead of just a plain linear model, you can use any machine learning approach. This includes powerful algorithms like random force and neural networks and other state of the art techniques.

4. The Naive Forecast

In this lecture, we are going to discuss a method of forecasting called the naive forecast before we discuss what the new forecast is. Let’s talk about the importance of establishing baselines in machine learning. The purpose of a baseline is to have a relevant point of comparison. To give you an example, suppose that you went to your professor tomorrow and say, hey, I just made a great discovery. I built a model using deep learning and I got 99 percent accuracy. Unfortunately, the statement is meaningless because you don’t have a context of a baseline. Your professor might say, but student, we already know that a simple linear model achieves 100 percent accuracy. In this case, your model is worse than what currently exists. And furthermore, it’s also slower. So how can we make sure that we don’t make this mistake? The answer is using a baseline. You’ll notice that when you’re reading machine learning papers, they often compare their results against the existing state of the art. Now, it’s important to realize that you don’t have to constantly try to beat the state of the art. In fact, that’s kind of antithetical to science because all you’re doing is chasing numbers. A good a real life example of that is when students study very hard to memorize their notes so that they can ace their exam without truly understanding the material or how it’s applied when you’re chasing only numbers. Sometimes it’s easy to forget why those numbers matter in the first place. Anyway, to get back to the point. It’s not always true that you want to compare to the state of the art. Sometimes if you just want to test whether or not something is working, for example, as a proof of concept, then the simplest model possible should suffice. In Time series analysis, the simplest model possible happens to be the naive forecast, what is the new forecast? Well, it’s just another name for something we’ve already discussed that is to copy the previous known value forward in time. One interesting phenomena that happens in a Time series analysis is that a lot of bad models end up looking like naive forecasts. When you look at the model predictions from afar, they seem to be pretty close to the true values. But when you zoom in, you can see that they seem pretty close only because they’re just copying the previous value or close to it. A really bad situation is this suppose that you fit a model and you say, aha, I beat the naive forecast because my accuracy is better than the Niyi forecast. But of course, you have to ask, are you talking about the accuracy on the sample data or the out of sample data? Note that in machine learning we often refer to these as the training data and the test data. So I would use these terms interchangeably. Just be aware. Well, one common mistake is that people believe because they got good accuracy on their in sample data, that the same will be true for their out of sample data. And also, quite commonly, it turns out that the opposite is true. You might beat the night forecast on the sample data, but on the out of sample data, the naive forecast beats you. Why does this happen? It happens when your model over fits to the noise in the training data, but doesn’t actually generalize to the true underlying pattern in the Time series if one exists. So that’s why it’s really important to compare your model to a baseline such as the naive forecast. It’s not good enough to say I got 80 percent classification rate on my train set and 75 percent on my test set. If some other method achieves 70 percent on the test, then your model isn’t as good. There’s one application I’ve seen a lot, which I think is a very interesting consequence of the times that we live in today. There are marketers everywhere on the Internet. The name of the game is SEO search engine optimization. Everyone is trying to get clicks for their blog or to get people to sign up for their course or whatever. And of course, one of the obvious topics that many beginners care about is stock price prediction. Now, if you’re taking this course, then you know better. But let’s continue the story. So imagine what happens. You take one of the most popular topics that would appeal to someone who doesn’t know about finance like stock predictions. You take one of the most popular machine learning algorithms that would appeal to someone who may not know a lot about machine learning, but has heard many buzzwords like LSM. LSM has been a very popular model for a sequence model. And then what do you do? Well, you combine these two, you make a blog post on stock predictions with LSD. In fact, many people have done so. It’s obviously a very appealing idea. No one would blame you for clicking on an article or buying a course that claims to be able to do this. I won’t name any names, but there are some courses that make the very mistake I’m talking about in this lecture. Maybe you’re watching this and you know where you’ve seen something like this yourself. There is another even worse mistake that these marketers make. But we’ll discuss that later when we talk about forecasting in general for a time series. Models like Arima, if you want, you’re encouraged to skip over to the forecasting lecture so we can continue this discussion. To get back to the naive forecast, let’s recall what we learned about random walks. As you recall, a random walk is where on every step of a time series I flip a coin or pick a number from a distribution. And that number is added to my current position in order to go to the next position. If my noise distribution is a zero centered Gaussian with variants Sigma Square, which is not unreasonable, then the best forecast is the naive forecast. I can do no better than predicting the last known value. Another way to think about this is if you build a model that you think is good but it cannot beat the night forecast, then it might suggest that your model is actually worse than a random walk model. In other words, a random walk model describes the data better than your model.

5. Simple Moving Average (Theory)

In this lecture, we are going to discuss a very simple technique and time series analysis called the simple moving average. The way it works is this. We start with a time series, then suppose we have some fixed length window. Then at each point in the Time series, we drag the window along and we calculate the sample mean of all the points in that window. This is the simple moving average. Let’s do an example. Suppose we have the numbers to zero eight three six one one six five five and let’s say a window is of size three. So let’s try to calculate the simple moving average. The first value is going to be the average of two zero and eight, which is three point thirty three. The second value is going to be the average of zero eight and three, which is three point sixty seven. The third value is going to be the average of eight, three and six, which is five point six seven. I would encourage you to try and calculate the rest of the values yourself in case you want a better understanding of how this is done. Note that we cannot fill in the first two values, since by definition we don’t have three values in which to calculate an average. Therefore, this works like the financial return where we leave the values we can calculate as an. So what is the purpose of the simple moving average, although it might seem extremely simple, it can actually be quite useful. For example, suppose that we need to calculate the mean and variance of a stock. We might use these values to characterise that stock’s distribution. In later sections of the course, you’ll see that these estimates are needed for portfolio construction. But we know that stock returns exhibit volatility clustering. So we have at two options. One, we can use all the values in our data set to calculate the mean and variance, or we can just use the most recent values specified by the window size. This might give us a better estimate since probably the return at six months ago or six years ago might not be relevant for the return today. Therefore, a moving average might be more useful than an overall average. In addition, learning the basic calculation behind the simple moving average also sets us up to learn about the exponentially weighted moving average, which then sets us up to learn a more complicated at times series forecasting methods. So think of this as a stepping stone to the more advanced methods in the section. Another interesting area, the simple moving averages useful, is with algorithmic trading, one simple strategy looks at the simple moving average of two different window sizes when these two signals cross. One way we use that as an instruction to take a long position when the two signals cross the other way. We use that as an instruction to take a short position. Since algorithmic trading isn’t a topic for this section, we won’t discuss this in more detail, but know that the simple moving average can be applied in more advanced areas. Let’s not discuss how the simple moving average will look in code for this, it’s convenient to use pandas in pandas. We use the rolling function to specify that we want to do some calculation on a rolling window. The input argument to this is the window size. Note that this doesn’t return the simple moving average right away, since there are actually multiple things you might want to calculate from the rolling window. In fact, this just returns that rolling window itself in pandas. This is a rolling object from the rolling window return value. We can calculate different kinds of statistics, such as the sum of the mean, the min, the max, the variance and so forth. If you have a rolling window of multiple columns, you can also calculate multi-dimensional statistics such as the covariance and correlation. In the next lecture, we will apply this code to actual data.

6. Simple Moving Average (Code)

In this lecture, we are going to look at how to work with the simple moving average in code. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend is once you know how this is done, to try and recreate yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As always, you can look at the title of The Notebook to determine what notebook we are currently looking at. So let’s start by downloading the file SP 500 closed at CSFI, as you recall, this is our CCV of closed prices only. Next, we’re going to import our usual library’s pendas matplotlib and non-pay. Next, we read in the CSFI as a data frame using Peaty to read CSFI. Next, we grabbed the clothes prices for Google, since we’re going to drop the any values, we want to make a copy first. Will assign this to the variable Googs X will do a Google ad head as a sanity check to see what’s in our data frame. Next, we’ll do a Google play it to see Google stock prices as a time series. Next will calculate the log returns for Google by calling the percent change function, adding one and then taking a log. Next, we plot the log returns as a time series. Next, we’re going to test our moving average in code, so on the right side, you’ll see that I’m going to grab the Googs column. Then I call the rolling function passing in 10 for the window size. Then I call the mean function to say that I want the rolling mean. Finally, I assign this to a new column called Esmay 10. Next, we call the head function with the argument twenty to say that we want to see the first 20 rows of our new data frame. As you can see, the first nine rows of semitone are in, and this is because no rolling window of size 10 exists for these elements. We can see that the first value in Smeeton shows up in the tenth row. As expected. You might want to do a manual test calculation to ensure that these values are what you expect. That is, calculate these values by hand. Next, I do another little sanity check to see what type of object is returned by the rolling function, as you can see, it’s an object of type rolling. This is always useful in case you need to look up the documentation. You know exactly which page to go to next. I call Googol Plot again so that we can see our new column plotted against the original time series. As you can see, it’s basically a smoother version of our original Time series, you might want to zoom in on your own computer or just select a few rows to see this more clearly. Next, we’re going to calculate another a simple moving average, but this time with the window size of 50. As you can see, this is the exact same code as before, except with 50 instead of 10, when we plot this, we see something very interesting. First, some 50 by using more values in its sample mean is even smoother than Smeeton. So your lesson is that the more values you use, the smoother the resulting time series will be. Second, you will see this lagging characteristic in the Time series. That is, it appears as if the simple moving average lags behind the original Time series. If you try different window sizes, you’ll see that this effect becomes more and more pronounced as the window size gets larger and larger. Next, we’re going to work with a multi-dimensional time series, let’s create a new data frame by using both Google and Apple stocks. Again, we make a copy and drop any any values. We’ll call this data frame Google Apple. On the next line, you can see that after calling the rolling function, I have other options than just the mean, since we now have two columns of data, I can calculate the covariance. As you can see, the results of this is pretty strange on the left hand side of this data frame, we can see a sort of multi level index. At the top level is just the date, but there’s a sub level where we get to choose either Apple or Google. So if you were to index this data frame by date, you would get a two by two covariance matrix. This is kind of a weird way of storing what is really a three dimensional tensor and a two dimensional data frame. Next, I demonstrate how you can select a single row by date and convert it to an umpire to get a single covariance matrix. Now, as you know, in finance, we like to work with returns rather than prices. So in the next block, we’re going to calculate the log returns for both Apple and Google. Notice that this is exactly the same code as before. Since it works for one or multiple columns. In the next block, we’re going to calculate the simple moving average for both Apple and Google returns. Notice that, again, it’s just the same code as before, except repeated twice. Next, we plot both Googles and Apples returns along with their moving averages. It’s interesting that there seems to be a pretty strong correlation between the returns, although there are definitely some outliers, recall that we discovered this in an earlier lecture as well. Next, we calculate the rolling covariance of the returns to do this, we call Rolling 50 and then we call the code function. If we do a code detail, we can see the last few rows of the covariance data frame. Notice that if I don’t pass in any arguments into the tail function, it just prints out the last five rows, which cuts off one of the covariance matrices. So this is a full covariance matrix. This is a full covariance matrix. And this is half of one. Next, we calculate the rolling correlation of the log returns to do this, we call rolling 50 again and then we call the core function. If we do a core detail, we can see the last few rows of the data frame I’ve passed in 16 so that we see the last eight correlation matrices and full. Notice how the correlation matrix changes over time for most of the correlation matrices, we can see that there is a negative correlation between Apple and Google returns, at least for these time periods. The final correlation is positive, demonstrating that the correlation can even change direction.

7. Exponentially-Weighted Moving Average (Theory)

In this lecture, we are going to discuss another kind of moving average, the exponentially weighted moving average. Note that some other names for this are exponential smoothing and the low pass filter. So if you’ve taken some of my other courses before and you’ve heard me use those terms, recognize that this is the same thing, in fact, that this kind of moving average is very applicable in many areas of machine learning statistics, finance and signal processing. So you will generally see it pretty often. So what is the exponentially weighted moving average? I want to break this lecture up into two parts, the first part is the short summary. If you want to only watch this part and then skip to the code, that’s fine. The second part of this lecture will be an optional in-depth discussion about why the exponentially weighted moving average has its name. You can opt to watch this if you want to get a better understanding of why and how this works. OK, so what’s the short summary, as you know, the arithmetic mean can be calculated by taking all of your samples, summing them together and then dividing by the number of samples, the exponentially weighted moving average is calculated differently. In fact, it’s calculated kind of on the fly or in an online manner. It says that the moving average at time T is equal to some constant alpha times. The sample at time T plus one minus alpha times the previous moving average at time at T minus one. In other words, at each step, the new moving average is the weighted some of the new sample and the old moving average. All right. So that’s pretty much it. It’s not a terribly complicated calculation. Of course, without further analysis, it’s not clear why this is an average and it’s not clear why it’s exponentially weighted. The next part of our short summary is this how do we do it in code similar to the simple moving average we call a function on our series or our data frame called IWM. This returns and IWM object, which is similar to the rolling objects we saw previously. It has a similar set of functions such as mean variance, covariance and so forth. To discuss a practical issue, what value of Alpha should we choose Alpha is something like a decay factor. Typically, Alpha is chosen to be a small value between a zero and one like zero point one or zero point to it might help to look at some extreme cases. So let’s say we choose Alpha equals one. That means set the average to be just the latest value of X.. In this case, all we’re doing is copying X and therefore it’s not really an average at all. On the other hand, let’s say we set Alpha equal to zero, then all we’re doing is copying the previous average and we’re not taking into account any new samples intuitively. Then if we set off a very close to one that says new samples matter much more in the old average matters, much less, you can imagine this will lead to a much more noisy time series which will more closely match the original if we set Alpha very close to zero that says new samples matter much less and the old average carries much more weight. In this situation, you’ll get a much smoother time series and it will take a much more drastic change in X to affect the moving average. OK, so now that the short summary is complete, if you want to know the details behind the exponentially weighted moving average, keep listening. Let’s suppose we want to calculate the usual arithmetic sample mean using the formula for the sample mean. You might suggest that this is quite obvious. Just take all the values of X that you’ve collected, add them all together and divide by the total number of X is that you have. The question is what’s wrong with this? I’ll give you a minute to think about it. So please pause the video until you think you have the answer. All right, so hopefully you thought about why calculating the sample mean naively might not be such a good idea. What if we have a lot or even an infinite amount of data? Obviously, our computers or our servers don’t have an infinite amount of space. And even if they did, calculating a summation is of T. So the more data you have, the longer it will take and that will increase linearly with how much data you’ve collected. Here’s my claim. I claim that you can make the calculation of the sample mean of one on each step in both space and time complexity, no matter how much data you collect again as an exercise before moving on to the next slide. I want you to think about how this might be the case. Please pause the video if you want to take a moment and think. OK, so hopefully you thought about how you might calculate a sample mean using constant space and time. The key is that you can calculate a sample mean using the previous sample mean let’s call the sample mean after collecting samples X bar subscript T, this means that the sample mean after collecting T minus one samples is X, bar subscript T minus one. We can write down the definition of both of these, which I hope is pretty obvious. Now that you know the metric, let’s again make this an exercise. Can you express Esbati in terms of X, bar T minus one, please pause the video until you’ve tried this on your own. OK, so here’s what you can do. First, you take Esbati and split up the summation so that you only sum up to T minus one. Then you leave X subscript T by itself. This is just the last sample you’ve collected. The next step is to realize that the sum of the ex towers from one up to T minus one can be expressed in terms of X bar subscripts, T minus one. We just have to rearrange the equation from earlier. It’s clear that this sum is just T minus one times X, bar T minus one. We can substitute this into our expression for Esbati to get the sample mean at time t in terms of the sample mean a time T minus one. One interesting thing you can do, although it’s not totally clear why you’d want to do this at this time, is split up the formula as follows. The first step is to multiply out the one over Tetum that gives us T minus one over T as the first coefficient and one over T as the second coefficient. The second step is to simplify T minus one over T to one, minus one over T. At this point we can just leave this as is this is the form that we want. We have one term with the previous sample mean and we have one term with the latest sample. What’s important to recognize about this equation is that we have discovered a way to calculate the sample mean that does not depend on carrying around all of the samples you’ve ever collected. All you need to have is the previous sample mean the latest sample and the number of samples you’ve seen in total. The next question to consider is, what if we believe that recent data matters more than past data? If we look at our equation carefully, we see an interesting characteristic. Remember that as we collect more and more samples, the value of tea is increasing. That means as we collect more and more samples, the weight that we give to the latest sample decreases. We can see that the weight that we give to the sample is exactly one over tea. Now, although this might make you think that the influence of each sample somehow decays over time, remember that this is not true because this is still just a regular arithmetic mean. But what if we want recent data to matter more, what would happen if instead of making the way one over tea, we simply make it a constant alpha? Well, then this is exactly the exponentially weighted moving average. The basic idea is instead of giving less and less weight to each new sample, we now give a constant weight to each new sample. Let’s see how this affects the influence of each sample overall. The next question we want to answer is, how does this update actually implement an exponentially weighted moving average? Can we show that this is true? And in fact, it’s not too difficult at this point. What we can do is just keep recursively plugging in. Older and older values of the sample mean so we can replace X, bar T minus one with its representation in terms of X bar at T minus two. Then we can multiply out the one minus alpha term so that we get X bar at T minus two by itself. Now we have three terms X bar T minus two, the sample at T minus one and the sample at time T. The next step is, of course, to replace X, bar T minus two with its representation in terms of X, bar T minus three. From there we can do the same thing, multiply out the one minus alpha and get each of the terms by themselves. At this point you should see a pattern. The number of individual samples keeps growing and the power on the one minus alpha term also keeps growing. If we keep repeating this pattern tee times, we end up with this expression involving a summation over all the past samples from one up to T. And of course, these weights are exactly exponentially decaying since Alpha is a number between zero and one, one minus Alpha is also a number between zero and one. And when you raise a number between zero and one to OPOWER, it gets smaller and smaller exponentially as K gets larger and larger. So how can we summarize what we’ve learned in this lecture, we’ve extended the concept of the mean to include the exponentially weighted mean. We can picture this by assigning weights to each of our samples with the arithmetic average. Each of the weights is just constant, with equal weight for each sample. With the exponentially weighted average, the weights decay exponentially, going backwards in time. This means that the latest sample matters the most. The second latest sample matters less. The third latest sample matters even less and so forth.

8. Exponentially-Weighted Moving Average (Code)

In this lecture, we are going to look at the code for the exponentially weighted moving average, this lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done to trying to recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So before we start to look at the code itself, let’s talk about the data set in this lecture and for most of the lectures in this section, we’ll be using the airline passengers data said. This is a famous time series analysis data set, and it’s basically the mists of time series analysis. So why do we want to use this data set instead of stock prices? Well, first of all, as you know, one of my main rules of machine learning is that all data is the same. That means even when you change your data set, this does not mean that your code or your algorithm changes. That’s why libraries such as the ones we use in this course can exist. Everyone, no matter what data set they are working on, can make use of these same libraries. So if you wanted to switch out this data set for a stock price data set, that is very easy to do. But recall what we said earlier about the random walk hypothesis and the efficient market hypothesis. Many of the techniques will be learning in the section on time series analysis depend on having some structure in the data that is more structure than would typically be observed in a stock price data set. Therefore, in order to really understand these algorithms, you want to achieve non null results and using better data sets lets us observe such meaningful results. Now, of course, you might wonder why do people even bother to teach these techniques in courses about financial engineering if they cannot be used in the same way that they are used for data sets like airline passengers? In fact, you’ll see that this is quite common. Many textbooks and classes which discuss statistical analysis for financial engineering look at these times. His analysis methods, as you recall, I said earlier that it’s not so much about making pretty predictions and it’s more about modeling and understanding how well your model matches the data. So a key part of this is mental. It’s getting past the idea of achieving high predictive accuracy. Certainly high predictive accuracy is motivating, and it’s a good indicator that your algorithm is doing what it’s supposed to do. And that’s why we’re looking at the airline passengers data set for the next few lectures. Furthermore, let’s recall that stock prices are not the only kind of data that one cares about in financial engineering. At more advanced levels, you’re going to realize, hey, actually trying to predict stock returns is actually a very small niche part of finance. There is actually so much more work that people in finance do. For example, you might have a job or your task is to forecast the cost of an engineering project. This will require you to do things like forecast sales numbers and other trends. So not looking at stock prices does not necessarily indicate that you are not doing finance, as you will see if you take a course purely on time series analysis. Many time series data sets look like the airline passengers data set. Remember that ultimately financial data is data about the world, not just technical data, which is just a very small part of all financial data. So to conclude, not looking at stock prices does not at all mean that you are not doing finance. That said, the first line here is to download our airline passengers CSFI. Next, we import pendas and numpty, not too surprising. Next, we use prediabetes, D.V. to load in our CSFI. Again, not too surprising. Notice that the indexed column is month. So this is monthly data rather than daily data. Next, we do a dot head to see the first few rows of our data set. As you can see, our index is the month and the passengers column contains the number of passengers in that month. Next, we do advocate is in L.A. to check if there are any missing values. Since the summer zero, there are no missing values. Next, we do a plot to plot our data said. So let’s spend some time analyzing what we are seeing, first of all, this looks nothing like a typical stock price time series. There seems to be very little randomness in this data set. In fact, that would mean that this is a great candidate for statistical in machine learning techniques. Remember that we used to call machine learning pattern recognition. That is, machine learning is all about recognizing and modeling patterns. And clearly there is a pattern here. On the other hand, stock prices might be characterized by their lack of patterns. We can see that with this data set, there seems to be a trend component and the cyclical component of the trend means that the signal is generally going upwards. The cyclical component means that there’s some periodicity where the same pattern repeats itself. We can see that this cyclical pattern actually amplifies over time, so it’s not a constant cyclical pattern. It’s also difficult to tell whether the trend is increasing linearly or if that is also increasing over time, perhaps exponentially in this Time series. We don’t have enough data to see that. However, one might surmise that many real world phenomena grow exponentially. For example, the number of transistors on an integrated circuit grows exponentially. This is known as Moore’s Law due to recent world events. We know that viral transmission exhibits exponential growth. Of course, this is limited at some point since the world population is finite. So in fact, the overall curve might look like a sigmoid. Let’s begin by setting our alpha parameter to zero point two. Why did I choose that zero point to no particular reason you can think of this as a hyper parameter that you can optimize? Next, let’s calculate the WEMA, this can be accomplished by first using the IWM function we pass in our alpha parameter and we say adjust equals false y adjust equals false. When you set adjust equals true, it actually does a different calculation than what we discussed in the three lectures, which is not what one usually considers to be the IWM. So we use adjust equals false so that Pendas uses the method we expect. Finally, we call the mean function because we want the exponentially weighted moving average, which corresponds to the exponentially weighted mean. We assign the result to the column called a. Next, just in case you’re interested in the data type returned by the EWTN function, I’ve printed that out. So as you can see, we get back an object of type IWM, so if you’re interested in looking at the documentation for this object now, you know which page to go to. Next, we do a plot to plot everything currently in our data frame, as you can see, the exponentially weighted moving average is kind of like a delayed and adaptive version of the original data set similar to the simple moving average. You might want to play around with the value of Alpha to get a better feel for how it affects the moving average. Next, what I always like to do if the equation is simple enough, is to check whether Pendas is doing what we expect it to do. That is, does it implement the formula we discussed in the theory lecture to check this? We’re going to use that formula manually to start. We’re going to create an empty list called Manual IWM. Next, we’re going to look through every element in the passenger’s column of our data set inside the loop. We’re going to check whether our manual IWM a list is empty or not. Remember that the IWM is calculated based on its previous value. If there is no previous value, then we can’t use the formula. Of course, that occurs on the very first value. The question is how does WMA initialize itself? In fact, there are many answers to this question you may have seen in my machine learning course is that the typical answer is to set the first value to zero. This induces a bias in the moving average because if, let’s say all your actual values are above one hundred, as they are in this case, the average is going to get pushed down from one hundred to zero as if zero were actually part of the data set. There is a way to correct this using a method called bias correction, which is used in modern optimization algorithms such as Atem. However, that’s outside the scope of this course for us. We will simply use the method of copying the first value. Otherwise, when the length of manual WEMA is greater than zero, we simply use our formula. We assign the result to exact and then we append X had to our list. Finally, when we are outside this loop, we assign our list to a new column of the data frame called Emmanuel. Next, we do a detailed plot again. We see that, indeed, our manual calculation appears to match what Panas does. Now you have a much better understanding of what Panis is doing under the hood. Not that we can also do a detached head to check how close the values actually are from the small print out, they appear to be exactly the same. Now, since these two columns are exactly the same, the next thing I’m going to do is remove the manual column since it’s redundant. This will help us clean up our data frame for the next few lectures.

9. Simple Exponential Smoothing for Forecasting (Theory)

In this lecture, we are going to bridge the gap between exponential smoothing and the whole winters’ model. This is going to be done in a series of steps. And the first step is to understand exponential smoothing from a different perspective. That is from the perspective of Time series forecasting. Previously, our perspective was simply what are different ways that we can take the average in a time series. Now it’s more about how do we fit a model to the Time series and then how do we use that model to forecast future values? Mathematically, this is the same as before, but philosophically it’s different. To do this, we’re going to have to introduce some new and more precise notation. Let’s begin by reviewing the exponentially weighted moving average once again, but using our new notation. So now we have we had a time t equal to Alpha times Y of T plus one minus Alpha times. We had a time T minus one in this form. Y had a time. T is our exponentially smooth version of Y at time t y it is the value from our time series at time. T Alpha is again the smoothing parameter. The next step is to now phrase this as a forecasting model, in this form, the equation looks as follows. You’ll notice that I’ve introduced this vertical bar symbol, which, as you know, in probability means given some on the left hand side, we’re saying why hat is the exponentially smooth forecast for a time T plus one given the known values at time t on the right hand side we have alpha times Y of T as usual, plus one minus four times the previous exponentially smooth value. Noticed something interesting about this, the time indices have changed for the current equation, the Y hat on the left is for the time index T plus one, the Y hat on the right has the time index T. If you look at the previous slide at the time and next is on the left and at T minus one is on the right. This is probably extremely confusing at this point. And I wouldn’t blame you for thinking so. It will make much more sense when we look at the code and you actually observe this behavior for yourself. Now, realize that I didn’t invent this. This is all part of the whole winters’ model. So don’t blame me if you find it confusing. By the time we reach the full whole Winsor’s model and you see it in action, you will see that this all makes sense. Again, recognize that this is now a forecast. Previously we were not forecasting, we were just calculating averages. The forecast for the next timestep is equal to Alpha Times, the currently observed value plus one minus alpha times, the current exponentially smooth’s fitted value. The next step is to express the simple exponential smoothing model in what is called component form. This will make it seem like overkill at this point because there is only a single component. However, the whole winters’ model contains multiple components, which is where this becomes useful. So in this form, the forecast equation looks as follows. We say that we had a time at T plus H given T is equal to live T, that is the forecast at an arbitrary eight steps ahead is equal to Helft, whatever that is. Then the smoothing equation says this elev T is just the exponentially smooth average of Y of t the Time series. Notice that L.A. now becomes the exponentially smooth average and Y hat becomes the forecast. Also, note that L.A. gets back its original time indices from the previous lecture, that is, we now have tea on the left and T minus one on the right. So left is itself not a forecast, but the forecast is based on liberty for this simple model. The forecast is simply assigned it to be a. But this will get more complex in later lectures. So what does Aloft represent? LTE is called the level, which will become more clear when we put it into context later on. Basically level can be thought of as the moving average, which kind of makes sense. The level is the average value of the signal in time, but the actual signal may fluctuate around that average level. One important thing to note about this forecasting model is that it’s not very expressive. The forecast is simply a constant value. It’s always left no matter how many steps ahead you want to forecast. This must be the case because remember, the exponentially weighted moving average is nothing but an estimate of the mean, since all we’re estimating is the mean. And that’s the only thing we can predict after we’ve stopped collecting data. We have no idea how this will change beyond the last known data point. And so our forecast simply consists of predicting the same mean value for each timestep. The last thing we were going to do in this lecture is talk about how this will work in code. Previously we looked at pandas as a way to calculate the exponentially weighted moving average. And we’ve noted that this lecture didn’t introduce any new concepts other than some renaming of variables. However, in the next coding lecture, we’re going to use stats models to perform exponential smoothing, which has a model that works more in line with what we discussed in this lecture. Furthermore, it’s capable of making forecasts so that will allow us to use exponential smoothing as a predictive model rather than just some way of taking the average. So how does it work, since this is the first model from stat’s models will be looking at in this course, how it works might be surprising if you come from a cyclone background. Basically, remember that this is a different library, so it has a different API. First, we start by importing the class, simple smoothing from the whole winters’ module inside stat’s models. Next, we create a model by instantiating an object of type, simple, smoothy. Note that when doing this, we pass in the data set, which should be a univariate time series, unlike Saikat Learn, which expects a two dimensional array, is input. This model expects a one dimensional array because the only dimension is time inside. It learn its number of samples by a number of features. Also, notice that, unlike Saikat, learn where the data is passed into the fit function with stats models, the data is passed into the constructor. The stats models object does have a fit function, which is what we will call next its function, except some parameters such as Alpha. So it’s kind of like the reverse of Saikat, learn in Saikat, learn at the constructor. It usually takes in the hyper parameters and the fit function it takes in the data in stats models, the constructor takes in the data and the fit function takes in the hyper parameters for us. We’re going to pass in Alpha into the fit function in addition to setting the argument optimized equal false. Note that for all of the models in the whole WINTERS’ module, it’s possible to fit parameters to the data. This makes it more like machine learning than simply calculating a moving average. But this will make more sense as our model gets more and more complex in later lectures. In order to make what we’re doing equivalent to what we did previously, we will not try to optimize the value of Alpha. Basically, Alpha would be optimized in order to minimize the error on the training set. If you’re familiar with machine learning, then you will understand that this involves taking the mean squared error between the true Time series and the predicted values and then minimizing that with respect to Alpha. If you’re not comfortable with this idea, it’s not necessary to know. So don’t worry about it since we’re not going to use it anyway. All right, so the fit function returns a result object, which is again different from what you’re probably familiar with from Saikat, learn in Saikat, learn you just get a reference to the model object itself. In stat’s models, we get a result object. Specifically, we get back a whole winters’ results object so you can look up the documentation if you’re interested in seeing what functions can be called on this object. The first function we are interested in is the predictive function, the way it works is this you pass in a start date for the start argument and an end date for the end argument. And then this will return an array containing a prediction of time series values for those dates. Note that these dates can be in sample or out of sample, meaning that they can be dates from the training set or after the training set, if the predicted dates are from inside the range of the train set, we call that an N sample prediction. If the prediction dates are from outside the range of the train set, we call that an out of sample forecast. A simpler way to get predictions is this if you want all of the train predictions, you simply call the attribute fitted values. This way you can avoid having to figure out the dates that you need for the start and end arguments. A simpler way to get a forecast is to simply call a forecast function. This accepts as input the number of forecasting steps. So, for example, if you want to forecast ten steps ahead, you just pass in 10. There’s no need to figure out what the data is. 10 steps ahead. Passing in the Integer 10 is enough. Note that this assumes that your forecast begins at the timestep after the end of the train set. So if your train set goes from one up to Big T, then your first forecast prediction will be for the time index. Big T plus one.

4. Portfolio Optimization and CAPM

1. Portfolio Optimization Section Introduction

In this lecture, we are going to begin the next section of this course, which is all about modern portfolio theory and the capital asset pricing model. This section will introduce you to some of the most fundamental ideas in modern finance, such as diversification, the Sharpe Ratio, the efficient frontier and the capital market line. To give you some idea of how this section fits into the rest of this course, I think it follows a very logical progression. First, we started with just trying to understand one of the core financial measurements, the return on an asset class. We asked what is the distribution of the return? How can we characterise that distribution in the next section? We started to look at the return as something that is time varying. We asked how does the return a change over time and can we predict those changes based on how it has changed in the past? One of the major lessons we learned during that exploration was that predicting returns is pretty hard. If we could predict stock returns, we wouldn’t need to know about portfolios in the first place. Logically, if we can predict the return of every stock, all we have to do is pick the best stock. Any portfolio would thus be suboptimal. In this section, we begin to accept the fact that this is not possible and that creating a portfolio of stocks is a more realistic solution. Let’s now outline what we will discuss in this section, we’re going to start by looking at probably the most famous portfolio there, S&P 500. This portfolio has not been created using the techniques in the section, but it is a real world example of a portfolio. Next, we’re going to define risk and we’re going to look at a concrete example of how diversification helps to mitigate risk. So this is one of the first concepts you will learn in this section. Risk is bad and creating a portfolio reduces risk. Hence portfolios are good. Next, we’re going to discuss how to characterize the portfolio in terms of two important quantities. It’s expected return known as the mean and its variants. The mean and the variance of a portfolio are extremely important quantities because what we are going to end up doing is called mean variance optimization. That is, we’re going to look at how to find a portfolio with the best mean and the best variance. Obviously, you don’t know what I mean by that. Yeah, but you will. We’ll be looking at two convex optimization techniques known as linear and quadratic programming. These techniques are what allow us to solve for the optimal portfolio. Given a set of constraints. A constraint is something that limits how you can invest. For example, you can’t invest more than 100 percent of your wealth. As mentioned previously, this groundwork will then lead us to some of the most fundamental concepts in finance, such as the efficient frontier, the Sharpe Ratio and the capital asset pricing model. This is yet another section of this course that became way more massive than I had planned, although I didn’t get to cover every topic I wanted to. I think this section goes over most of the topics that I think are fundamental to financial engineering. Thanks for listening and I’ll see you in the next lecture.

10. Maximum and Minimum Portfolio Return in Code

In this lecture, we are going to look at how to find the maximum and minimum return of a portfolio in code. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Let’s start by importing the Lehnberg function from Zippi. Next, let’s get D the number of assets in our portfolio. This is equal to the length of our Mean Return series. Next, let’s set HQ and BQ, these are defined as per the previous theory lecture, so hopefully you took notes and you recognize why these should be set this way. Next, we’re going to set the bounds to be consistent with the other parts of this notebook, we’re going to set a lower limit of minus zero point five with no upper limit. The bounds argument should be an iterable of length D and each item inside the iterable should be another iterable containing two items the lower bound and the upper bound for each variable in order. Now that we have all our input set up, we can call the line Praag function, the first argument is the C vector that goes into the objective will do minimization first. So this is just the mean return vector. Next, we pass in IQ, BQ and the bounce. So let’s run this. OK, and we get a message saying that the optimization terminated successfully. So what else from this output is useful for us to look at? We can see from the fun item that the smallest return is about minus zero point one nine five. We can also see from the exciton the optimal value of the weights. These are set the first, second and last asset to the lowest possible weight, which is minus zero point five to set the third asset to the highest weight possible, which is two point five. Notice that these sums are one. Now, note that there are other ways to check that the optimization was successful under status. We will get zero if the program was successful and we will also get true under success. So you can check either of these and you can check the message. And this will all tell you whether or not the program was successful. Sometimes your linear program may not be successful. For example, if you pass in constraints that are impossible to meet as an example of that, suppose you say you want your rates to be greater than one and less than zero. Of course, it’s not possible for a number to be greater than one and less than zero simultaneously. In the next block, we obtain the optimal value of the objective by calling resident Fine will assign that to a variable called Min Return. Next, we can run a linear program to find the maximum return since Lin Praag does minimization by default, all we need to do is negate the mean return vector. So let’s run this. OK, and we see that this has also terminated successfully. Now, you might find this output strange, the optimal value of our objective looks to be minus zero point one eight. But how can that be if we are maximizing the return? Surely we had stocks that had positive return. In fact, this is because we just optimized the negative of the return. Therefore, the negative of this is the actual return, which is positive. Zero point one A. So in the next block, we set Max return to be negative Redfearn fun. Next, we print out the min return and the max return of one last time. So approximately minus zero point two and plus zero point two.

11. Mean-Variance Optimization

In this lecture, we’re going to continue our discussion on how to optimize the portfolio. Previously, we looked at a simple problem where we wanted to take a set of assets and find the maximum and minimum possible return of any portfolio containing these assets. We found that this problem could be solved using linear programming. Of course, those portfolios didn’t take into account risk in this lecture. We will consider how to optimize the portfolio in terms of risk and return simultaneously. Note that there is an inherent tradeoff in portfolio optimization. We know that as expected, return goes up, so does risk. So it’s not possible to say give me the maximum return, but the lowest risk. As long as you want to increase your return, you have to be willing to take on more risk. Portfolio optimization is all about not taking any more risk than is necessary. We’ve seen examples of suboptimal portfolios. For example, consider these two points. They both have the same risk, but one has higher return. Obviously, if you are willing to take on this amount of risk, it’s better to obtain the highest amount of return possible for that level of risk. Now consider these two points. They both have the same return, but one has less risk. Obviously, if we really wanted to have this expected return, we will choose the portfolio with less risk. There’s no reason to take on more risk than we have to. So that’s the idea behind portfolio optimization. We want to trade off risk and reward, but we don’t want to take on any more risk than necessary. And we also want to maximize our reward for a given level of risk. As you recall, the way that we measure risk is by standard deviation in this lecture, we will focus on a variance because as you know, this gives us a nice quadratic function, which is typically easy to optimize, find the gradient of and so on. Also, since the square is a monotonically increasing function, if we minimize variance, we’ve simultaneously minimized the standard deviation. All right, so knowing what we know so far, this problem is quite easy to formulate if we make some assumptions, let’s assume that our objective is to minimize risk or in this case, variance. That is, we want to minimize transpose sigma times W with respect to W, let’s also assume that we would like to achieve a specific expected return on our portfolio. So we have the constraint that dotted with W is equal to R some desired return. Furthermore, we still have the constraint that the WS must come to one which makes our portfolio a valid portfolio. Finally, we may have some additional constraints like all the wise greater than or equal to zero and less than 30 percent. You will notice that this problem is very similar to our maximum and minimum return problem, it’s still a constrained optimization problem. We still have the same constraints for the most part, with one additional linear constraint for the desired return. But the main difference is that the objective function is now quadratic instead of linear. In fact, when we have a quadratic objective with linear constraints, we call this a quadratic program or cupie for sure. Now, unfortunately, there is no functioning Sipi for quadratic programming. This is very strange because there is a quite Praag function in both R in Matlab, so it seems appropriate for Sibai to have one also. So what can we do about this? Well, there are several options. One option is to use a library called KVI Zapped, which is a library for a convex optimization available in Python. We will not focus on this option, although you may come across an example or two. The second option is to stick with ZIPPI and use a generic optimization function call to minimize technically if this is suboptimal because it’s designed for generic functions. If you know your problem is a copy, then you can use more specific algorithms to solve the problem using its structure. So we would expect the performance of KVI except to be better since that handles quadratic programs explicitly. However, using Skype is easier because you don’t have to manually determine all the matrices to pass in to your cupie solver. Later, we will look at another optimization problem for maximizing the Sharpe ratio, which does not have an obvious linear or quadratic structure. So it’s nice to have a generic optimizer like MINIMIZE, which can handle anything. So for this reason, we will focus mostly on using MINIMIZE, which will require a bit less thinking about the implementation. And also you don’t have to install SVCs OPT, which is probably less familiar to you than Zippi. Before we move on to the lecture, I want to give you a brief rundown on how the minimized function works. The first arguments of the minimize function is a function. This is the function that you actually want to minimize for us. This will be the variance. So if you have a function that takes on a set of weights for your portfolio and returns the variance, that would be exactly what you need. The second argument to the minimize function is X not which represents the initial guess for our weights. We will use the equally weighted portfolio, which is just the uniform distribution over each asset. The next relevant argument is method sipos. Minimize function can use different optimization algorithms, but we will be using one called Sless Cupie. This stands for sequential least squares quadratic programming. Now don’t be fooled by the name. This is still a generic optimizer. The next relevant argument is the bounds. This works exactly the same way as it did for the line function. So you can pass on a list with the items and each item will be a tuple containing the upper and lower bound for that element. Finally, we have the constraints argument. We’ll get into this later in the lecture. But basically this is represented as a dictionary or list of dictionaries where each dictionary holds a constraint. You can either tell it to be an equality or an inequality constraint, and then you can pass in a function that returns a specific value to tell zippi whether or not that constraint is met. Now, I want you to pay attention to the fact that this does not require you to pass in any specific matrices or vectors like the covariance or mean return vector. All you’re doing is passing in functions. This means that Zippi doesn’t know what the parameters of your problem actually are, which on one hand makes this easier for you to write. But on the other hand, it makes it less optimal than using a piece of a directly.

12. The Efficient Frontier

In this lecture, we are going to expand on what we learned in the previous lecture on mean variance optimization, in the previous lecture we looked at how to set up the mean variance optimization problem, and we talked about how you would go about solving it in code. This lecture is meant to take a more a big picture perspective. What do we actually get when we solve this problem and how can we use the results? Let’s recall what the quadratic program we derived earlier actually gives us. It says that given a specific target return and given some particular constraints on the weights, tell me how I can allocate my portfolio so that I can achieve the smallest possible variance. Let’s see what that means in terms of the risk return plot. Earlier, we drew a risk return plot showing random portfolios. We saw that these points seemed to fill out a pretty well-defined curve called the Markowitz bullet. Let’s consider the question, what does mean variance optimization have to do with this curve? We can begin by considering how we might apply mean variance optimization. First, recall that we know how to find the minimum and maximum possible return, given a set of assets for that, we can use linear programming. So let’s say we’ve done that and we found the min return and the max return. Now, let’s suppose we take small steps from the minimum to the maximum, say 50 or 100 steps, so, for example, we might start at minus two percent and then go up to minus one point nine percent, minus one point eight percent and so on. For each of these possible returns, we will consider them the target return. Then for each of these target returns, we will find the minimum variance portfolio for that return. This will give us the best possible risk level for each of these returns. Basically, you should understand that by doing this, we will draw an outline of this curve. Why do we get an outline? Well, consider any point inside the curve. We know that these points cannot possibly be optimal, since all we have to do is move to the left to achieve less risk. Solving the quadratic program we derived earlier will automatically give us the leftmost point corresponding to the minimum risk for the given return. The top part of the curve we have just drawn has a name, it is called the Efficient Frontier. Any portfolio along the efficient frontier is called efficient. Why are they efficient? Because for each of these portfolios, if you take the expected return that they have, there is no better portfolio that will give you less risk. Conversely, you can look at it from the risk. First, you can say for each of these portfolios at a given level of risk, you can find no better expected return. Thus these portfolios are efficient. Now, you might ask, why don’t we consider the bottom part of this curve? These portfolios are not efficient because all we have to do is go up and we found a portfolio with the same risk, but with higher return. For the rest of this lecture, I want to discuss a few alternative ways that you can find the efficient frontier using constrained optimization. Let’s start with a simple one. What if we want to get rid of the bottom half of the curve, the part that is not part of the efficient frontier? In this case? It’s simple to reformulate our quadratic program. All we need to do is replace the equality constraint on the target. Return with an inequality we say that might transpose W must be greater than or equal to the target return rather than just equal. In this way, the quadratic program is allowed to minimize the risk with any larger return. You can imagine that this will push the solution upwards to the point where we have minimum variance. Previously, we looked at the scenario where you want to start with the target return and optimize the weights such that they minimize risk, but what if we wanted to look at this the other way? What if we want to start with the target risk and then figure out how to maximize the return? As a portfolio manager, you can imagine how this would be highly practical. A claim may come to you and say, I don’t want more than five percent risk. What is the highest return you can give me? Well, we can do that as well. In this case, we want to maximize value, transpose W with respect to W, this corresponds to the portfolio return. This is subject to the constraint that the portfolio variance is less than or equal to the target variance. As usual, we have our constraint that the weights must come to one and other optional constraints on the weights. Note that this is not a quadratic program because we have a quadratic constraint. A quadratic program has quadratic objective and linear constraints. However, this problem is convex. If you want to learn more about these kinds of problems, I would recommend learning about convex optimization. Yet another formulation of the problem is this let’s suppose that we want to be able to tune the risk relative to the return. Our objective is now the portfolio return minus some hyper parameter tau multiplied by the variance of the portfolio. We want to maximize this objective. If you have a machine learning background, this should remind you of the concept of regularization. We are essentially penalizing the expected return by adding a cost for the amount of variance that the portfolio incurs towers called the risk aversion parameter. And of course, this allows you to specify how important the variance is relative to the return. Of course, time must be positive so that your penalty is positive. As usual, the weights must come to one and you can have additional constraints on the weights. Note that because the objective is quadratic and the constraints are linear. This is another instance of a quadratic program. OK, so to summarize this lecture, here’s what we’ve done. First, we define the efficient frontier and we discussed what it means to be efficient. Next, we looked at alternative ways of finding the efficient frontier. The first way was to use the quadratic program we formulated in the previous lecture. This is to minimize variance, given a target return. The second way was to do the opposite. We can maximize return given a target variance. The third way was to introduce a risk aversion parameter where we maximize the return penalized by the variance.

13. Mean-Variance Optimization And The Efficient Frontier in Code

In this lecture, we are going to look at mean variance optimization in code. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. As you recall from the theory lecture, what we would like to do is walk through every possible return in a very small steps from the minimum return up to the maximum return. For each of these points, we will find the minimum possible risk, since these steps will be very tiny. When we plot them, they will look like a nice, smooth curve outlining the efficient frontier. So let’s start by setting an equal to 100 points. This is the number of target returns that we will calculate the minimum risk for next. We call the line space function to get 100 equally spaced points between the return and the max return. Next, we’re going to import the minimize function from Zippi. Next, we’re going to define some functions to pass into our optimizer. The first function will be our objective, which should return the variance of our portfolio. This function should only take in a single argument which represents the portfolio. It’s this function should return a single output, which is the portfolio variance. As you recall, this is computed by taking the weights, dotting them with the covariance and then dotting the result with the weights as a side note. Since we’re just using a generic optimizer, we probably could have taken the square root of this as well, since there’s no reason our problem has to be a valid quadratic program. Next, we’re going to write a function to represent the constraint on our target return. This takes into arguments which are the weights and the target return. This will be used as an equality constraint. The way it works is it should return zero when the constraint is met. Therefore, we first calculate the portfolio return by taking the DOT product between the weights and the mean return vector. Then we subtract the target. If they are equal, then this function will return zero. Next, we have a function for our portfolio constrain, that is the sum of the weights must equal one. Again, we have an equality constraint, so this should return zero when the constraint is met. We can accomplish that by taking the sum of the weights and subtracting one. If the sum of the weights is one, then this expression will return zero. Finally, we’re going to create a list of constraints to pass on to our optimizer. This list should contain dictionaries with each dictionary representing a separate constraint. So the first constraint here is the target return constraint. As you can see, the type argument specifies the kind of constraint it is. This can either be equality or inequality. In this case, it’s an equality. So we pass an IQ. Next, we have the funky, which takes in the function of representing the constraint. Next, we have the Arctic, which takes in any arguments that should be passed into the function after the optimization variable, as you recall, our function takes into arguments the optimization variable, which represents the portfolio weights and the target return since the target return is the only extra argument. This is what we need to specify here. Note that each time we call our optimizer the target, return will be different, as you recall. We’re going to step through a whole bunch of target returns to draw the efficient frontier. So this element of the dictionary will be updated on the fly inside our loop, which you will see later. Next, we have the equality constraint for the sum of the weights. This one is much easier. This takes in no arguments. So we just have to say the type is equal and fun is our portfolio constraint function. Next, we’re just going to run our minimised function once to see that it works. So the first argument is the function we want to optimize, which is the portfolio variance. Next, we have X not which represents our initial guess for the weights. This is an iterative algorithm. So this is where the algorithm starts from. We use the uniform distribution, which is a vector of size D containing one already for each element. Next, we have method, which is ASCAP. Next we have the constraints argument where we can pass in the constraints list that we made earlier. So let’s run this. And we see that the optimization was successful. Most of these values you don’t need to worry about, the important ones are fun, which tells us the minimum variance we were able to achieve and X, which tells us the actual weights which achieve this variance. By the way, notice how large these weights can get. If we look at the X-ray, we can see that the first asset gets shorted over 100 percent. This is probably too extreme, so we should set the bounds as well. In the next block, we’re going to run our function again, but this time we’re going to pass in the same bounds that we passed into our linear program earlier. So let’s run this. And our optimization has terminated successfully. Notice how now the weights don’t go below minus zero point five. Also notice our variance is now 11 something. Previously it was 10 something, if you remember. So what happens when we limit the amount that we can short sell is that there are some areas of the weight space that are inaccessible. Previously, we had to short over 100 percent to achieve that minimum variance. And now, since we are no longer allowing the program to short that much, our variance will be higher. OK, so next, we’re going to run through all of our target returns to generate the efficient frontier. So let’s start by creating an empty list to store our optimized risks. Next, we will live through each targeted return inside the loop. We’re going to set the constraint dictionary to the current target. Next, we’re going to call the minimized function with exactly the same arguments as before. The difference now is that the constraints are being updated on each iteration. When the loop is done, we’re going to appends the square root of the optimize value to our optimize risks list. This is because we optimize the variance and the square root of that is the risk. Next, just in case we’re going to check the status of the result, zero means success. So only if the status is not zero, we will print out the result just in case something went wrong. OK, so let’s run this. And since we didn’t get any printouts, it means nothing went wrong. Next, we’re going to plot our efficient frontier. Let’s start by drawing a scatterplot of all our random portfolios once again. Next, we’ll call Palsied apply to draw a line chart on the x axis. We’ll have the optimize risks. And on the Y axis, we’ll have the target returns. We’ll color this black. So the line is differentiated from the scatterplot. OK, so let’s run this. And as expected, we see that the line chart we generated traces out the efficient frontier, plus the negative side as well.

14. Global Minimum Variance (GMV) Portfolio

In this lecture, we are going to discuss one noteworthy portfolio, which is the global minimum variance portfolio or the GMV for short. This is the portfolio, which is at the very tip of the Markowitz bullet. It’s the portfolio out of all possible portfolios. That is the minimum variance. One reason you might be interested in the GMV is because trying to predict the expected return is difficult in modern portfolio theory, all the work is done under the assumption that, you know, the expected return and covariance. But in reality, you have to estimate these. We’ll discuss this more later in the section. But the basic idea is the average return is less stable than the covariance. Because of this, the covariance is easier to estimate in order to solve for the GMV portfolio. We only need the covariance and we can completely ignore the expected return. Therefore, we don’t have to worry about the fact that it’s hard to estimate. Let’s see how. OK, so the minimum variance problem is defined as follows, we want to minimize the portfolio variance subject to the constraint that the sum of the weights is one. Additionally, we can add other constraints on the weights to limit how extreme they can be, as usual. However, note that there is one thing missing here, which is the expected return. In other words, Mbewe does not appear. Why don’t we need this? This is because the global minimum variance portfolio doesn’t care what the return is at all. It’s only objective is to minimize the variance, as you can see from the problem.

15. Global Minimum Variance (GMV) Portfolio in Code

In this lecture, we are going to look at how to find and plot the GMV portfolio in code. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so let’s start by calling the minimize function. Once again, you might want to double check that we have almost all the same arguments as before. Our objective is get portfolio variance. Our initial guess is that uniform distribution, our method is cupie. Our bounds are set to what they were before. The only thing that’s different is the constraints argument where we only pass in a single constraint, which is the portfolio constraint. That is, all the weights must sum to one. So let’s run this. OK, so our optimization has completed successfully. Next, let’s find the minimum risk by taking the square root of the minimum variance. We also need to find the corresponding return. So let’s first obtain the GMV waits by calling Residex. Next, we can calculate the return by dotting these weights with the mean return vector. Next, let’s try our risk return. We’re going to start by drawing a scatter plot of our random portfolios, although this is not completely necessary at this point. Next, we’re going to plot the efficient frontier the same way we did earlier. Finally, we’re going to make another scatterplot with just a single point representing the GMV that we just found will color this red to differentiate it from the other points. OK, so as expected, the GMV is located at the very tip of the Markowitz bullet, which is also at the very bottom of the efficient frontier.

16. Sharpe Ratio

So let’s discuss more about how to compare different portfolios, we know so far how to find portfolios along the efficient frontier. These portfolios are optimized in the sense that for any given return, we can’t find a portfolio with lower risk and for any given risk, we can’t find a portfolio with higher return. So we know there’s a trade off between the two. As the return goes up, so does the risk. But how can we compare it to different portfolios which are both efficient? Is there any notion of better in this case? The answer is yes. In order to reduce each portfolio from two numbers into a single number, we simply take the ratio between the two. That is, we take the ratio between the return and the risk. To be more precise, we take the portfolio expected return minus the risk free rate divided by the portfolio standard deviation. This is known as the Sharpe Ratio. It is named after William F. Sharpe, who won the Nobel Prize in economics. Now, although we haven’t discussed the risk free rate up until now, it should be pretty intuitive if you have ever invested before. There are certain financial instruments you can purchase, which will guarantee you a certain return since its guarantee that there is zero risk. We’ll discuss this more very shortly. Going back to the Sharpe ratio, we can imagine the numerator as the excess return. It tells us how much more return we can achieve as a reward for taking on more risk. Another way to think about the Sharpe ratio is it’s the excess return per unit of risk. As an example, suppose we would like to compare two portfolios. Portfolio A has an expected return of 12 percent and a volatility of 10 percent. Portfolio B has an expected return of 20 percent and the volatility of 20 percent. In both cases, the risk free rate is one percent. We can calculate the Sharpe ratio for Portfolio A as follows. It is twelve minus one divided by ten, which is one point one. We can calculate the Sharpe ratio for Portfolio B as follows. It is twenty minus one and divided by twenty, which is zero point nine five. Therefore, Portfolio A has a higher Sharpe ratio. Although it’s absolute expected, return is lower, it’s excess return per unit of risk is higher. Therefore, Portfolio A has a higher Sharpe ratio. OK, so now that you understand the basics of the Sharpe ratio, let’s discuss the risk free rate a bit more. As mentioned previously, if you have ever invested with your bank, you are probably familiar with financial instruments which are risk free. In the United States, for example, you can purchase a certificate of deposit, also known as a CD, which gives you a guaranteed return on your investment, of course, as per our observation that there is a trade off between reward and risk. You can imagine that the return on these is quite low compared to riskier investments in practice. We usually use the interest rate on the current three month Treasury bill, also known as a T bill. Note that this is the case in the United States, so it may be different in your country. Another practical note worth mentioning is that because Treasury bill rates have been so close to zero over the past few years, people often just set it equal to zero. In this case, the Sharpe ratio is just the expected return, divided by the standard deviation. Also, keep in mind that interest rates are usually provided in annual terms, that is it’s the interest rate you would see if you invested for an entire year. So if the interest rate on a three month T bill is one percent, this does not mean that you will get one percent return after three months. It means you would get one percent return after one year. However, in code, since we are working with daily returns, we will divide at the annualized tibial rate by 252 to get the equivalent daily interest rate. So on the topic of annualizing, one thing worth mentioning is that the Sharpe ratio is often presented in annualise form. We won’t bother to do this since it’s just a constant. But I want you to be aware that this is basically the way you do this is you take the number of measurements you would have made in a year. You take the square root of that and you multiply by the Sharpe ratio that you measured. For example, if you’re working with daily data, then you would take the square root of two 52 and multiply that by your Sharpe ratio. If you’re working with monthly data, then you would take the square root of 12 and multiply that by your monthly Sharpe ratio. However, keep in mind that although this calculation is common, it’s actually approximate and depends on quite a few assumptions. If you want to learn more, check out the paper called The Statistics of Sharpe Ratios by Andrew Low in extra reading that 60. Not that this makes sense in the context of annualizing your expected return in covariance matrix to begin with, if we annualize the expected return and covariance that’s just multiplying by two 50 two, if we have daily data after multiplying by two 50 to we have their annual expected return and the annual covariance. When we calculate the Sharpe ratio, we’ll have to 52 times the daily return divided by the square root of two, 52 times the variance. 252 divided by the square root of 252 is just the square root of 250 to. So now that we know what the Sharpe ratio is and what it means, we can discuss how to calculate it in code. In fact, this is not much different than what we’ve already been doing. Since our minimized function works for minimizing any function, we can just pass in a function that returns the Sharpe ratio almost directly, I say almost, because if we want to maximize the Sharpe ratio, we need to negate our expression. Therefore, we’ll want to write a function that returns the negative Sharpe ratio. Luckily, we already know how to calculate each item inside this function in the portfolio return is just the mean return dotted with W and the portfolio variance is just W dotted with the covariance dotted with W, and because we would like our results to be consistent with all the work we did previously, all the other arguments for this function will be the same as before. So one benefit of using a generic optimizer like Sipos Minimize is that we can maximize the Sharpe ratio directly without having to worry about what kind of program it is. In fact, it doesn’t seem to fit into any one of the usual forms, such as linear programs or quadratic programs. In actuality, you can convert the Sharpe ratio maximization problem into a quadratic program. The details are outside the scope of this course, but I’ve left a link in extra reading dot text in case you want to check it out.

17. Maximum Sharpe Ratio in Code

In this lecture, you going to find the portfolio with the maximum Sharpe ratio in this lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So let’s start by figuring out what is the risk free rate? Basically, you can just Google this and you will find a link to it. St. Louis Fed Doug. From there, you can look up the relevant data and find the three month tibial rate for that day. OK, so I chose zero point zero three because that was the approximate value for the data we are looking at. Note that this is not three percent, but in fact, zero point zero three percent. In other words, this is a very tiny value and this justifies why many people approximate this value as zero. However, I think it’s useful to go through the full calculation. Note that since our data is daily, we also want to divide by two fifty to to get the effective daily interest rate. Remember that although the table rate is for three months, the rate listed is an annual rate. OK, so next, we’re going to write a function that returns the negative Sharpe ratio, why the negative Sharpe ratio? This is because Zippi has a minimized function and not a maximize function. So if we want to maximize the Sharpe ratio, we have to minimize the negative Sharpe ratio. OK, so inside this function, we first calculate the mean and standard deviation using the usual formulas. Next, we take the mean, subtract the risk free rate divide by SD and negate the result. Next, we’re going to call the minimized function to find the maximum Sharpe ratio. So for the fun argument we’re going to pass in the function we just defined nagu Sharpe ratio. Other than that, you should be familiar with all these other arguments by now. So let’s run this. OK, so the optimization terminated successfully, by the way, notice how easy that was, if we were going to use quadratic programming, we would have to go through the work to derive the quadratic program and figure out all the matrices that go into the cupie solver. Instead, we can just pass in our function directly and not have to worry about the problem having any specific structure. Next, we’re going to assign the maximum Sharpe ratio and the corresponding weights to variables called at best S.R. and Best W. Now, you recall that earlier we did an experiment where we generated a bunch of random portfolios. In fact, this is another method of estimating the solutions to the various problems that we’ve saw so far. You can imagine that after generating enough portfolios, you will have generated nearly all portfolios. And so if you want to find the maximum or minimum something, you can just directly search through all those random portfolios to see which one is optimal. Of course, this is only approximate since you would need to generate an infinite number of portfolios to get the true answer. However, it’s worth trying to see how close we can get. By the way, this is known as the Monte Carlo approach, which is why I’ve prepared the variables here with me. OK, so to start, I’m going to initialize the best weights to none and the best Sharpe ratio to minus infinity. Next, I’m going to answer a loop to iterate through all the risks and returns that we generated earlier. Next, I’m going to calculate the current Sharpe Ratio S.R., which is the return minus the risk free rate divided by the risk. Next, I check whether this new Sharpe ratio is better than the best Sharpe ratio we have found so far. If it is, we save the Sharpe ratio and we save the corresponding weights. When we’re done, we print the best weights and we print the best Sharpe ratio. In the next block, we are going to plot everything we just found, so to start, we’re going to first draw scatterplot of all the random portfolios, we’ll use an alpha of zero point one so that these portfolios will be transparent. Next, we’ll plot the efficient frontier as we did before. We will color this black. Next, we’re going to plot the portfolio with the maximum Sharpe ratio found by your optimizer. Note that we don’t yet have the corresponding risk and return. However, we can calculate these using the weights that we saved. We’ll call these ops risk and ops red. So next, we draw a scatterplot with just this one point and we’ll color it red. Next, we do the same thing except for the portfolio found by a Monte Carlo simulation. We’ll color this pink to differentiate it with the previous point. OK, so let’s run this. And what do we see? We see that the portfolio with the maximum Sharpe ratio falls along the efficient frontier. So that’s a good sanity check if at this point we’re not on the efficient frontier. That would mean we got the wrong answer. Next, we can see that the Pink Point, the portfolio found by Monte Carlo is pretty close to the portfolio found by direct optimization. In other words, if for some reason you couldn’t use optimization, Monte Carlo will give you a reasonable answer.

18. Portfolio with a Risk-Free Asset and Tangency Portfolio

In this lecture, we are going to consider a new scenario. Previously, we’ve been assuming that we only invest in risky assets. However, we know that we could potentially invest in the risk free asset, which has no risk. This lecture will consider the question what happens when we split our investment into both the risk free asset and the portfolio with the maximum Sharpe ratio for those of you who do not like math. I’m going to start by telling you the answer. And for those of you who do like math and you want to know how and why we arrive at this result, you can listen to the second part of this lecture as well. OK, so what happens when we split our investment between the risk free asset and the risky portfolio with the maximum Sharpe ratio, what we get is a line. This line connects to the point where we have a zero risk and return are CBF the return of the risk free asset. That makes sense because if my return is our Sabbath, that means I have zero risk. This corresponds to the case where I’ve invested nothing into the risky portfolio and everything into the risk free asset. The other point that this line connects to is the point where we find the maximum Sharpe ratio. This also makes sense. This is the point where we’ve invested everything into that risky portfolio and nothing into the risk free asset. And I’m sure you don’t need to be convinced that both of these are achievable. We already know how to find the portfolio with maximum Sharpe ratio, and we already know how to achieve zero risk by investing only in the risk free asset. All you need to do is buy a tibial. The interesting part of this is the points in between what we will find is that these points are also achievable, but what do these points represent? If you think about it, these are points where we can obtain a higher return with the same level of risk. How do we only invested in risky assets? Furthermore, we can obtain even less risk than the minimum variance portfolio and still achieve higher return compared to the minimum variance portfolio. So it seems that including a risk free asset is pretty useful in a risk return plot. We know that going in the direction of the upper left is always better. That means we’re getting higher return and lower risk. We see that this line represents portfolios which achieve exactly that. Another interesting point about this line is that it can extend above the efficient frontier. How can this be? Let’s think about it this way. The point on the left, what we have zero risk means that we invest zero percent into the risky portfolio. The point on the efficient frontier with maximum Sharpe ratio means that we invest 100 percent into the risky portfolio. Therefore, we can go further than this if we invest more than 100 percent into the risky portfolio. How can we achieve that? We can achieve that by shorting the risk free asset. This is equivalent to borrowing money so that you can invest more into the risky portfolio. So one interesting consequence of this line is that it is tangent to the point where we find the maximum Sharpe ratio for this reason, we call this portfolio the tangency portfolio. It simply means that the efficient frontier is tangent to the line we just found at the point where we find the maximum Sharpe ratio. We know from calculus that tangent line means that this line has the same slope as this point on the efficient frontier. Therefore, we can also say that any portfolio made up of the tangency portfolio and the risk free asset all have the same Sharpe ratio. In other words, the maximum Sharpe ratio is also the slope of this line. Any portfolio found on this line has that same Sharpe ratio. OK, so in the rest of this lecture, we are going to derive everything that we just discussed. We can start by considering what we want to find. Basically, we’re creating our portfolio out of portfolios. This portfolio contains the maximum Sharpe Ratio portfolio with only risky assets and the risk free asset. We would like to find the mean and variance of this portfolio. In fact, this is very similar to an exercise we did earlier in the section where we considered how to characterize a two asset portfolio. The first asset is the tangency portfolio and the second asset is whatever gives us the risk free rate. OK, so let’s say that the tangency portfolio has a return of Sebti expected return immutably and various segments of T squared. The risk free asset has returned RCF and Variance zero. Note that our is known. So it’s not random. OK, and next, we would like our new portfolio to be some combination of the above two. So let’s say our SAPI is equal to Sabeti Times, our Sebti plus one minus WAPT Times, our Sabbath. Sebti tells us how much to invest in the tangency portfolio and one minus WSP tells us how much to invest in the riskless asset. OK, so what is the expected return of our new portfolio, if we take the expected value of both sides, we can see that we get Mississippi equals to Sebti times Musab TI plus one minus WUFT Times. Arcibel, this one should be pretty easy. Now, what about the variance? We can start by taking the variance of both sides since the tangency portfolio and the risk free asset are independent. The variance of the sum becomes the sum of the variances. This is because there is no correlation, however. Also note that the risk free asset is not actually random and its variance is zero. Therefore, the variance of the mixed portfolio is just T squared times the variance of the tangency portfolio. Since everything is squared, we can take the positive square root of both sides to get a simple linear relationship between the standard deviations. If we rearrange this a bit, we can see that the weight of T can also be expressed as the ratio of the two standard deviations. As before, we would like to replace WCT so that we can find some equation for the portfolio return in terms of the portfolio risk, that is, we want to know how the return varies with risk because that’s exactly what we are plotting when we have a risk return plot. OK, so essentially we want to plot Mississippi versus Mississippi. Therefore we have to find a Mississippi as a function of Sigma SAPI. Since we know that support is equal to Sigma Suppy over Sigma Sebti, we can just plug that in. The next step is to multiply everything by Sigma Sebti so that there are no more ratio’s. The next step is to move all the terms involving Sigma Sebti to the left and to move other terms involving Sigma Sarpy to the right, after doing so, we can factor out Sigma Sebti and Sigma Sarpy. The final step is to move Sigma’s city back to the right side by dividing everything by Sigma’s Sebti, once we’ve done that, we can just add RCF to both sides. This gives us the equation that you see here. So what is interesting about this equation, if you look very closely, you can see that it’s linear. There are lots of similar looking symbols here. So you have to pay close attention to what the variables are. The variable for the X axis is the portfolio risk sigma suppy. The variable for the Y axis is the portfolio return muser. So that’s why I formatted this equation in a very specific way. It’s in the form Y equals M, X plus B or in other words, sloper times X variable plus Y intercept. We see that the intercept is the risk free rate. This occurs when the risk is zero. Of course, that makes complete sense. This is when we’ve invested everything into the risk free asset class. Furthermore, we see that the slope of this line is the Sharpe ratio of the tangency portfolio. It’s the excess return divided by its standard deviation. This means that any portfolio made up of the tangency portfolio, plus the risk free asset, has this specific Sharpe ratio. We can also see this if we rearrange one of our earlier equations in a slightly different way. Now we have the Sharpe ratio of the mixed portfolio on the left in the Sharpe ratio of the tangency portfolio. On the right, we can see that this equality always holds. Therefore, the Sharpe ratio of the mixed portfolio is always equal to the Sharpe ratio of the tangency portfolio.

19. Risk-Free Asset and Tangency Portfolio in Code

In this lecture, we are going to apply more portfolios, these portfolios will incorporate the risk free asset and therefore will form a line on the risk return plot. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so this lecture is going to be pretty short. First, we start out by again drawing all the random portfolios which will fill up the inside of the Markowitz bullet. Next, we draw the efficient frontier using the color black. Next, we plot the tangency portfolio, which is the portfolio with the maximum Sharpe ratio. As you recall, we save the weights for this portfolio in a variable called the best W we can use best W to calculate the corresponding risk and return, which we will call ops risk and ops red. Next, we draw scatterplot with this single point and we will color it red. Next, we will draw our attention line, as you know, in matplotlib, the way that we draw a line is that we connect x y coordinates together. The first x y coordinate is the point where we achieve zero risk. Thus X one is zero and Y one is the risk free rate. The second x y coordinate is the point where we invest only in the tangency portfolio. Therefore X is ops red and Y two is ops risk. Next, we call the plot function passing in the values of X one, x two and Y one, y two that we just calculated. So let’s run this. And we can see that our line does, in fact, look like it is tangent to the efficient frontier as an exercise, you can try extending this line beyond the tangency portfolio. This will correspond to the scenario where you borrow cash at the risk free rate so that you can invest more into risky securities.

2. The S&P500

In this lecture, we are going to look at one pretty popular portfolio, the S&P 500 note that the S&P 500 does not use the techniques we will discuss in this section. However, it does introduce a few key terms. It’s important because it’s a real world example of a portfolio and many people use it as a proxy for the market. So let’s suppose you have one thousand dollars to invest, you might say, I want to put three hundred dollars into Apple, two hundred fifty dollars into Google, two hundred fifty dollars in a Starbucks and two hundred dollars into IBM. Note that because you have one thousand dollars to invest, these amounts must add up to one thousand. In other words, three hundred plus two fifty plus two fifty plus two hundred is equal to one thousand. Notice how if we divide everything by one thousand, we will get fractions on the left side and we will get one on the right side. These fractions. Tell us what percentage of our wealth has been invested in each security, for example. This says that I’ve invested 30 percent into Apple. Twenty five percent into Google. Twenty five percent in a Starbucks and 20 percent into IBM. In portfolio theory, we call these percentages weights. Note that weights must sum up to one or equivalently 100 percent. So what are the weights for the S&P 500? The S&P 500 is called a cap weighted index, that is the weights for each company are proportional to that company’s market capitalization. If we call each weight WCI, it is equal to the market cap of company I divided by the market cap of all companies, some together. You should be able to convince yourself that this sums up to one. So what is market cap in the first place? Market cap is simply the number of shares of a company that exist multiplied by the share price. So pretty simple to calculate. Another question you might have is how are the five hundred companies in the S&P 500 chosen in the first place? In fact, these companies are selected by a committee that is by people. They have several requirements, such as a minimum market cap, minimum trading volume, and that the company must be listed on the New York Stock Exchange or NASDAQ.

20. Capital Asset Pricing Model (CAPM)

In this lecture, we are going to very briefly discuss the capital asset pricing model, also known as the CAP. Luckily, we’ve already done all the theory necessary to understand the CAPM. The CAPM is really just a philosophical discussion and a renaming of some variables in the equations which we’ve already discovered. So what do I mean by that? Well, earlier we looked at the portfolio with the maximum Sharpe ratio, we call this the tangency portfolio. Why? Because when we invest in this portfolio, in addition to a risk free asset, we get a line tangent to this point on the efficient frontier. So they cap em, simply renames these items. The tangency portfolio is called the market portfolio. Furthermore, this line is called the capital market line or the CML, when we talk about the market portfolio. We say that it has returned R7. The expected return is therefore Amusa them and its corresponding risk is Sigma Sivam. So again, all we’ve done is rename the variables. So what’s the philosophy behind this renaming? Why is the tangency portfolio, the market portfolio? Well, this all stems from the efficient market hypothesis, which, as you recall, we first encountered in our discussion of Time series forecasting. We assume that every investor has the same data and every investor is a mean variance optimizer. Because of this, they have to choose the tangency portfolio with some mixture of the risk free asset simply because there is no better portfolio. This is the portfolio that gives you the maximum Sharpe ratio because the tangency portfolio is held by all investors under this model, then this is the market portfolio. Let’s look at an example of how one might use the capital market line. Suppose that you’re considering investing in a project to build Silicon Valley’s next disruptive technology. Let’s say that the price of one share today is one thousand dollars, and this is expected to be worth twelve hundred dollars in one year with volatility 20 percent. The current interest rate RCF is three percent. The expected return on the market portfolio is 15 percent and the volatility of the market is 10 percent less. Determine whether or not this investment is worth considering. We can calculate the expected return of the investment as twelve hundred divided by one thousand minus one, which is 20 percent. But according to the cap m, if we take on a risk level of 20 percent, we should expect the return to be three percent plus 15 minus three over 10 times 20 percent, which is equal to 27 percent. Remember, this is the equation for the capital market line. So what can we conclude? We can conclude that this investment is not worth considering. According to the Sharpe ratio of the market, we should demand a return of at least 27 percent on an investment with 20 percent volatility. We would expect more reward for the given level of risk. There is some other interesting points about the cabin which are worth discussing, although we won’t go into the derivation right now. There is another line called the security market line, which tells us the return on each individual asset. The basic idea is it is a linear regression of the assets excess return on the market’s excess return. That is the expected return of asset ie minus the risk free rate is related linearly to the return of the market, minus the risk free rate. Note that the excess return of asset AI is also called the risk premium, it’s how much more return you can get for taking on a certain amount of risk. The excess return of the market is also sometimes called the market premium. This relationship between the risk premium and the market premium is controlled by beta, by the coefficient, it tells us how much asset it moves relative to the market. If beta is greater than one, we consider this an aggressive investment. That means you’re willing to take on more risk in return for more reward if beta is equal to one. We would say that this asset has average risk if beta is less than one. We would say that this asset is not aggressive because of this. In the capital, asset pricing model beta is actually the measure of risk. This might seem contrary to what we discussed earlier where we said that Sigma was the measure of risk. So that might seem confusing, but rest assured, everything is consist in large beta means more risk in small beta means less risk. This is called systematic risk or market risk because its risk there, as you can see from the equation, depends completely on the market. It is also unpredictable, assuming the markets are efficient, since the market itself is unpredictable. Another interesting fact is that when we plot the security market line, the independent variable is Beita rather than a market premium, that is, it’s a plot of the return versus beta. According to the cap em, all assets should fall on this line. If they do not, then that suggests those assets are mispriced. This should help you understand why we call Beta the systematic risk. It says that the risk premium systematically increases or decreases as you adjust beta the risk. When your risk goes up, your return goes up. When your risk goes down, your return goes down. So the CAPM says that all assets should fall in the security market line, but let’s be clear that this is an assumption of the capital asset pricing model. This is part of the model. But this lecture is not an assertion that the model is true. Only if you believe the model is correct would you believe that this is true. Another point worth repeating is that although this equation seems quite random, it can actually be derived from the modern portfolio theory that we just discussed, it involves a little calculus, so it might be best left for another lecture. Intuitively, though, you can recognize that this equation has the same form as the equation we derived earlier, which showed that the Sharpe ratio of any portfolio on the capital market line is the same as the Sharpe ratio of the market portfolio. If we combine both the segments together into a single constant, we see that the excess return of any portfolio is proportional to the excess return of the market portfolio. So what is the purpose of the security market line and the capture? If we look very carefully at the name of this model, it’s called the capital asset pricing model. That is to say, it’s a model that helps us to price an asset. In other words, we can estimate under the assumptions of this model what the price of an asset should be. If we find that an asset is mispriced, investors can use these signals in order to make investment decisions. If an asset is undervalued, that would be a signal to buy. This assumes that the market will eventually correct itself so that the asset goes to its correct price. If an asset is overvalued, that would be a signal to sell. There is yet another line called the security characteristic line, which is similar to the security market line, but adds a time index note that we also take away the expected values and replace them with an explicit noize term. When we do this, this is what we get. So while the Capitol hours are random variables, note that all of these terms depend on time except for Beita, which is assumed to be CONSED in. Another way to represent this is to replace the excess returns with a single symbol. We’ll say our prime is the excess return of our. So now our equation becomes r.i prime equals beta sabai times are in prime plus epsilon. A more elaborate model involves adding an alpha term to our previous equation. Note that according to the CAPM, the expected value of Alpha is zero, which is why it hasn’t appeared until now, including Alpha allows us to account for the possibility of mispricing. As you recall, when we fit a linear regression, we get both a slope and an intercept. Thus, Alpha represents the intercept. It tells us how much return cannot be attributed to the market. In statistical packages such as R stats models, when you fit a linear regression, you also do various statistical tests, including ones to check whether or not your slope in your intercept are statistically significant. So after you fit your model, you’ll get a P value telling you whether or not Alpha Zibi is significantly different from zero violating the assumptions of the CAPM. So what is the interpretation of Alpha if we find that Alpha is significantly different from zero, then the security is mispriced, at least according to the assumptions of the Kabam. If we find that Alpha is greater than zero, this means that the security is underpriced. In other words, the returns have been at too large on average. This would be a bicycle. However, it’s important to recognize that if we reject the null hypothesis that Alpha is equal to zero, all we have done is shown that the security was mispriced in the past. There is no guarantee that this will hold true in the future. As a summary to this lecture, it’s worth noting that the capital asset pricing model is most certainly wrong. Let’s consider some assumptions made by the capital. Number one, it assumes that the markets are in equilibrium. In other words, for each asset, supply is equal to demand. This is not true. Number two, it assumes that every investor has the same forecast of expected returns and risks. Again, this is most certainly not true. Number three, it assumes that every investor is a mean variance optimizer and every investor holds a combination of the tangency portfolio and the risk free asset. Again, not true. Considering these facts, it may be surprising to you that the CAPM is still tied to financial engineering students around the world. In fact, the CAPM is still a useful pricing model, as shown by the example earlier in this lecture. It is still a fundamental topic of financial engineering and it forms the basis for more advanced techniques. In fact, if you continue your studies in financial engineering, one of the topics you will learn about is factory models. The Cap M is considered to be a single factory model because the return on each asset is completely determined by a single factor, which is the market. Fama and French developed a model called the three factory model, in which the return on each asset could be regressed on three separate factors, one of which is the market. Furthermore, one can use statistical methods such as PCA and factor analysis to obtain what are called statistical factor models. These are unsupervised machine learning methods which try to figure out what the factors might be without you having to assign them explicitly.

21. Problems with Markowitz Portfolio Theory and Robust Estimation

Now, although modern portfolio theory is a staple of financial engineering, there seems to be one glaring hole in all of this. And I’m sure you’ve been wondering this yourself, how do we actually know the expected return and covariance in the first place? In fact, we don’t. As one might expect, this lack of knowledge has a huge effect on the effectiveness of these techniques to general strategies. One could apply to estimate the expected return and covariance are as follows. First, you can use historical returns to calculate the sample mean of the return and the sample covariance, for example, one might use the simple moving average or the exponentially weighted moving average. Although this sounds pretty bad, it’s actually not uncommon in some of the more advanced techniques we will touch on actually build on this. Second, you can use machine learning to predict what the expected return and volatility may be in the future. Note that this may include all kinds of data, including fundamentals, data, public data and private data, which we discussed earlier in this course. As always, if you go this route, it will be very important for you to test whether or not those predictions are better than the baseline. As before, I want you to take a more nuanced and mature view of this problem, what is our strategy here? If you’re from machine learning, usually your strategy will be to ask, how can I make my predictions more accurate? Now, I want to be clear. This is still an important question to ask. It is also the case that many people underestimate how difficult this is. And by this point in the course, I think you are getting used to going beyond this simple question. At this point, we are accepting that we will be wrong. Being wrong is nothing but a guarantee. The real question is how can we mitigate the effects of being wrong? That is to say, how can we limit the impact of having incorrect estimates? Can we measure this impact? In the rest of this lecture, I will outline several strategies that one can use to answer these questions. The first technique I want to discuss is one we’ve already seen, which is the global minimum variance portfolio, as you recall, this is a portfolio that tries to minimize the variance without any regard to the return. The reason why this works is because, as it turns out, the sample return is much less stable than the sample covariance, knowing this one might want to simply remove the return from the problem completely by only minimizing the variance. Note that one interesting consequence of this is that it can actually lead to good returns. Another technique we’ve talked about is setting bounds on the portfolio weights. This includes not allowing short selling and also limiting the maximum value of the weights. One interesting observation you can make while doing mean variance optimization is that it usually selects pretty extreme weights. We often see weights that suggest investing 100 percent or more in single assets. This kind of goes against the idea of diversification. You can imagine that if you decide to trust these estimates and invest in such extreme portfolios and you turn out to be wrong, your potential to lose money will be much greater. In fact, this is an experiment you could easily do yourself. One method of measuring the effect of estimation error is to use bootstrapping in bootstrapping, we pretend that our sample is the true data. We then re sample from this sample to simulate what happens when we sample. That probably sounds confusing. So let’s do a simple example. Suppose that my returns are just one, two, three, four, five. Let’s suppose that we have any data points so that end equals five. A sample of this would be to draw in samples from this set of samples with replacement. An example of what that might look like is one, one, two, two, three. Another example is one, two, three, three, one. Because we sample with replacement, duplicates are allowed. If duplicates were not allowed, you can imagine that it wouldn’t make much sense because we would just get one, two, three, four, five each time. OK, so then what do we do? Let’s say we do this sampling process B times B might be a large number, like one thousand each of these B times we calculate the optimal portfolio from the sample returns. Of course, these portfolios are not truly optimal because we didn’t use the true mean and covariance. And remember, in this setup, the so-called true, mean and true covariance are just the mean and covariance of the original sample. Then we can see how far off these estimates are from what they would have been. How do we use the original sample? So what is the result of bootstrapping? This is a box showing what we get when we use bootstrapping with and without short selling in this plot. The horizontal dash line represents the true Sharpe ratio of the tangency portfolio. On the left, we have a box plot showing the results when short sales are allowed. On the right, we have a box plot showing the results when short sales are not allowed inside each plot. We have two columns. The left column shows the actual Sharpe ratio achieved and the right column shows the estimated Sharpe ratio. So what can we infer from this? First, we can see that in both cases, whether or not you allow short selling, we tend to overestimate the Sharpe ratio by quite a substantial amount. In other words, in each box by the right column tends to be above the dotted line. On the other hand, are actual Sharpe ratios are obviously worse than the maximum achievable Sharpe ratio. In other words, in each box plot, the left column tends to be below the dotted line. However, notice the difference between a short selling and not short selling. When you allow short selling, not only do you tend to overestimate more, you also tend to get worse actual performance when you don’t short sell. You still don’t achieve the maximum Sharpe ratio, but you get much closer and that’s the end goal. After all, we want the portfolio. We guess we should have to be close to the portfolio we should actually have. The last technique I want to mention is shrinkage estimation. The basic idea behind this is instead of using the unbiased estimates of the mean and covariance, which we know can be error prone, we will mix these with some other more stable estimate. If you’re familiar with Bayesian machine learning, map estimation and other similar ideas. This should feel pretty intuitive to give you a taste of how this works. Consider the sample covariance. Let’s call that sigma hat unbiassed. Instead of using sigma hat unbiassed, we will take Delta Times, some highly structured estimate called F.. We will add this to one minus Delta time Sigma hat unbiassed. So our shrinkage estimate of the covariance becomes a mixture of these two. How we estimate F is outside the scope of this course. But if you’re interested, you’re encouraged to read the paper titled Honey, I Shrunk the sample covariance matrix, which you can find an extra reading by text. As before, we find that using shrinkage estimates allows the Sharpe ratio we achieve to be closer to the true maximum Sharpe ratio and as usual, we consider Delta to be a hyper parameter so you can use the usual methods for hyper parameter selection. As mentioned previously estimating the expected return is much more difficult than the covariance for the expected return, one commonly accepted method is the black Lierman method. This uses actual Bayesian techniques in order to estimate the expected return briefly instead of obtaining a point estimate of the expected return. We actually try to measure its posterior distribution. Interestingly, this method combines the data with the subject of views of the investor. That is, it accepts human input. This method is far beyond the scope of this course. However, if you’re interested in this topic, I would recommend checking out the article titled A Step by Step Guide to the Black Letterman Model in extra reading texte.

22. Portfolio Optimization Section Conclusion

In this lecture, we are going to conclude and summarize everything we learned in this section, this section was all about how to create a portfolio. Why do we want to create a portfolio? Because it’s a very bad idea to try and predict each stock one by one and invest in what you believe is the best stock. Most likely your prediction will be wrong and such an investment will be very risky. You stand to lose a lot of money. This section was all about how to diversify your investment such that you can mitigate this risk. In this section, we learned about how to find the expected return and variance of a portfolio, we looked at the two asset case and also the asset case. We noted that in the two asset case, no portfolios appear inside the efficient frontier. That is to say, if you have two efficient portfolios, then all combinations of those two portfolios are also efficient. We learned about the risk return plot and how to populate it with random portfolios. We noted that random portfolios are often suboptimal. We looked at how to optimize portfolios in terms of various quantities. We learned how to maximize and minimize return using linear programming. We looked at how to minimize variance using quadratic programming. We learned how to plot the efficient frontier. We learned about the Sharpe ratio, a quantity that relates the return of a portfolio to its risk. We learned how to find the portfolio with the maximum Sharpe ratio on the efficient frontier. We learned that this is also called the tangency portfolio. We learned how to find other portfolios that are more efficient than purely risky portfolios by including a risk free asset. We learned about the capital asset pricing model and how that makes use of the modern portfolio theory. We previously derived finally we looked at the problem of robust estimation and how we can mitigate the effects of having incorrect estimates of the true mean return and covariance.

3. What is Risk

So in this lecture, we are going to begin to explore a new topic, which is portfolio optimization and modern portfolio theory at this point, I think we spent quite a bit of time getting comfortable with one particular fact, and that is stock returns are random. It’s similar to how a coin toss is random. We don’t ask what is the result of the next 10 coin toss is going to be. We know intuitively that this is a silly question. A better question to ask is how our coin toss is distributed and are some coins correlated with other coins? And can we exploit these relationships in order to optimally allocate our resources? That’s what this section is all about. This particular lecture will discuss the concept of risk in finance, there are several ways to conceptualize risk, but in this course we will be focused on one, which is the standard deviation. Now, at this point, we all know what standard deviation is. So there’s no need to define it again. We’ve seen it many times in this course already. It’s the stigma that we see in the normal distribution, although, of course, most distributions have a standard deviation. The question for this lecture will be why is this an appropriate measure of risk? Previously, we looked at how you might try to predict stock returns, you might think of your prediction as the expected value of the stock return at some point in the future at this point. We know that those predictions will be wrong to some degree. The question isn’t, will you be wrong because we know that we are guaranteed to be wrong. The question is how wrong? Suppose that I predict my expected return will be 10 percent. But let’s see, the realize return is nine percent. In this case, that’s not so bad. But what if I predict my expected return will be 10 percent and the real return is minus 20 percent? Now, that is very bad. As you know, the standard deviation characterizes how far away the samples of a random variable can deviate from its meaning. And so you can see there’s a direct relationship between that statistical concept and this financial concept of how bad your prediction of the return will be. Let’s do some simple exercises to test your intuition about risk. Consider this scenario where you have a choice between two stocks, stock and stock B. We can see that they both have the same volatility, but the return for stock is higher than the return for stock. B as an easy quiz question, I want you to think about which stock you would prefer to invest in A or B.. I’ll give you a minute to think about it, so please pause the video and come back when you are ready. OK, so hopefully you chose a stock A has a higher return, and therefore if you invest in the stock, you will make a larger profit. Let’s now consider a different scenario this time stock and stock B have the same return, but different volatility. Again, I want you to take a minute to think about which of these you would prefer, given the choice. Please take a moment or pause the video until you’re ready. OK, so hopefully you chose stock B, stock B has a lower volatility, but the same return in this scenario, we prefer stock B because there is less opportunity to lose money. Remember that in reality, you don’t know when these wild swings in price are going to happen because they are in the future. So avoiding high volatility is generally desirable. Now, let’s consider a third scenario in this case, stock has a higher return, but it also has higher volatility. Still, you have to make a choice between either A or B, so which one do you choose? Again, please take a moment to think about this and pause the video until you think you have the answer. All right, so in this case, there isn’t enough information to answer this question, and unfortunately, scenarios like this are exactly the kind of scenarios we encounter in the real world. As you’ve seen previously, it’s generally the case that higher return also means higher risk. Sometimes it may come down to a personal decision. Maybe you just don’t like taking on risk. It could be due to what stage you are in in your life. It’s generally advise that younger people invest in riskier funds because if anything bad happens in the market, they can afford to wait for it to recover. However, there are many different funds to choose from, each with varying levels of performance and associated risk. Some funds, which are often called target retirement funds, are automatically allocated based on the date that you plan to retire. They generally follow the rule that in the early years, more volatility is acceptable in exchange for higher rewards and in the later years they avoid risk at the cost of also having lower expected returns. OK, so to summarize this lecture, why do we want to avoid risk? Well, let’s just compare two time series of stock prices with one stock price is just a constant linear trend upward. We can buy at any time and sell at any time. And we are making a constant amount of profit per unit of time. We invested in that asset. But now let’s take Starbucks stock price. There is lots of variability, hence lots of risk. Now, you might look at this and think, well, there’s a strong trend upward. Surely I would have made money. But remember, you are not that smart. You didn’t predict that you should buy Starbucks stock on this day. And so on this day, rather, it’s much more likely that you would let your emotions guide your decision making. Imagine if you buy on this day. Why? Because you see Starbucks stock going up very quickly. You think, wow, this is a great investment. We’ve all been there. Then you see Starbucks stock drop rapidly. You think, oh, no, I better cut my losses. So you sell and then the stock goes back up. So you made completely the wrong decision. Remember, you can’t predict the future because if you could, you wouldn’t be taking this course in the first place. So recognize that although there is a general trend upward, there are still many opportunities to lose money with a risky asset.

4. Why Diversify

In this lecture, we are going to discuss the concept of diversification. Now, I’m sure you’ve all heard of this before. Diversification is good. Diversify your income stream, diversify your business. But why? What advantage does this offer? People often say don’t put all your eggs in one basket. This is pretty intuitive. If you put all your eggs in one basket and you drop your basket or your basket is stolen by a bandit, then you will have lost all your eggs. In other words, putting all your eggs in one basket is risky. As you recall, risk is what we discussed in the previous lecture. In this lecture, we look at how diversification can limit risk. Let’s again consider a scenario where you have two choices. There are, again, two stocks, stock A and stock B. This time both stocks have the exact same return and the exact same a standard deviation. Let’s suppose for simplicity’s sake, that the returns are both normal with me and Mu and Sigma squared. Importantly, let’s also suppose that the returns for A and B are completely independent. As you know, independence also implies no correlation. So the question is this. Should you choose to invest only in stock, only in stock B or half in stock and half in stock B.. I’ll give you a minute to think about it, so please pause the video. And so you think you have the answer. OK, so let’s look at how to answer this question first, note that whatever choice you choose, you will always have the same expected return. If you only invest in stock, your expected return is HMU. If you only invest in stock B, your expected return is also HMU. If you invest half in stock and half in stock B, then your expected return is zero point five times more plus zero point five times mbewe, which is still MUE. But what about the risk again, we will use standard deviation as a proxy for this, the variance, if you only invested in stock A is sigma squared. If you only invest in stock B, the variance is still sigma squared. The variance, if you invest half in stock and half in stock B is a little harder to calculate, but you should be able to prove each step yourself. Let’s start with the answer and then we can talk about how it’s calculated later for those who are interested. So the answer is that the variance of your investment, if you put half in stock and half in stock B is only one half sigma squared. That is the standard deviation is sigma divided by the square root of two. So what have we shown, we have shown that even if we have two stocks with the exact same expects a return and the exact same variance, and even if investing equally in both stocks yields the same expected return, it is better to invest in both because it results in smaller risk. The variance goes down by a factor of two and consequently the standard deviation goes down by a factor of two. This is just another example of not putting all your eggs in one basket because the stocks are assumed to be independent. If one stock goes down, it says nothing about what will happen to the other stock. In this optional part of the lecture, we’re going to circle back and show why the variance of the split investment is reduced by a factor of two. You probably want to be comfortable with this kind of calculation, since basically all of portfolio optimization has to do with calculations just like this. OK, so let’s start by saying X one comes from a normal distribution with me, Ammu and Variance Sigma squared, X two also comes from a normal distribution with me mu and variance sigma squared. Let’s also write down that X one and next to our independent. Otherwise these calculations would be more complicated. Next, let’s say we have a new random variable Y, which is half of X one plus half of X two. Our job is to find the variance of why there are two ways of doing this. The quick ways this recall that the variance of a constant see times a random variable is C squared times, the variance of that ran a variable. You can prove this yourself using the kind of math I’m going to show you very shortly. Furthermore, we know that if two random variables are independent, then the variance of their sum is just the sum of their variances. So in effect, we get that the variance of Y is one half squared times sigma squared plus one half squared times sigma squared. The result is one 1/2 times Sigma squared, as mentioned earlier. Therefore, we have shown that by splitting up your investment equally between two independent stocks with the same distribution, we have reduced our risk. The slow way of finding the variance of why is this so for those of you who don’t want to take my word for any of the identities from the previous derivation, this is more of a step by step process. OK, so let’s start by writing out the definition of variance. It’s the expected value of Y minus the mean of Y all squared. Note that the mean of Y is just mu, which is the mean of our two assets. Note also that there is an alternative expression for the variance, which is easy to derive. The first step is to expand the square, we get Y squared minus to Y times, MU plus minus square. Next, we separate the expected value of each term. Note that the two in the MU are constants so they can be brought outside the expected value. Next note that the expected value of Y is just the mean of Y, which we know is MU. So we get the expected value of Y squared minus two times MU squared, plus B squared. Of course that’s just the expected value of Y squared minus MU squared. OK, so this is a more helpful expression of the variance for the purpose of this lecture. Next, let’s replace Y with one half X one plus one half X two. And again, the next step is to expand the square. We then get a quarter X one squared, plus a quarter X two squared, plus a half times X one X two. OK, so the next step is, again, to separate out the terms and the expected value. As usual, the constants can be brought outside. The interesting part of this step is that X one, an X two are independent. Therefore the expected value of X one at times X two is the expected value of X one times the expected value of X two. As you know, these are both MU. So in the next step, we just get one half mile square, we can combine the two squared terms together to get minus one half square. The next step is to factor out the one over four note how we get to Times Square, the next step is to simply give one of the few square terms, tax one and to give the other musical term to X to. Of course, this is just the variance of X1 plus the variance of X2, as you know, these are both sigma squared, so we get one quarter time sigma squared plus Sigma Square. And of course, that’s just one half sigma sware. And again, just in case you forgot what we were doing, we are showing that the variance decreases when we invest equally in two different independent assets. This is even though they have the same exact expected return and the same exact risk. Putting your eggs in multiple baskets decreases your risk.

5. Describing a Portfolio (pt 1)

In this lecture, we are going to look more in-depth at how one describes the portfolio mathematically, basically this is going to consist of a few definitions in some statistics. As you’ll see, this will be necessary to optimize the portfolio, which is the goal of this section. OK, so as you know, a portfolio is a set of weights, which tells us the proportion of each asset in our portfolio. For example, let’s suppose we have Apple, Google, Starbucks and IBM stock under consideration. Let’s also suppose that we have ten thousand dollars to invest. Let’s say we decided to use twenty five hundred dollars to purchase Apple stock. Five thousand dollars to purchase Google stock. Fifteen hundred dollars to purchase Starbucks stock. And one thousand dollars to purchase IBM’s stock. In this case, our corresponding awaits would be zero point two five zero point five zero point one five and zero point one. In general, our weights are described by a vector w w subi tells us the proportion of our money we’ve invested in Asset I in this class. We’ll see that I goes from one up to be. This means that we have these assets in our portfolio, not that we have a constraint on W since the vector W contains percentages, it must be the case that the sum of all the WS is one. Sometimes we have an additional constraint that each use of I must be greater than or equal to zero. In fact, this constraint is quite common. Now, you might ask, how can I possibly be less than zero? How can we have a negative percentage invested in an asset? This is a case that we call short selling. We’ll discuss this more later in the course. But for now, it suffices to understand that a negative weight corresponds to short selling. It essentially means that you are selling an asset which you do not own and then buying it back at a later date. Your hope is that the asset will go down in price so that you can buy it back for less than what you sold it for originally. Note that this is the opposite of what you want with an asset that you would actually buy. If you actually buy a stock, you want it to go up in price so that you can sell it at a later date for more than what you bought it for. In either case, whether you allow negative weights or not, it’s still must be that the sum of the weights should be one. OK, so what’s next at this level, portfolio theory is concerned with the returns over a single period, we’ll call the return on asset IE in this period our SABAI. As you know, this return is a random variable. Therefore it has a distribution, it has a mean and as a variance and so on. The mean of return, our sabai is just the expected value of our Sabai. We use the symbol mutebi to denote the mean as we usually do. Note that if we combine all the mutual buys into a single vector called Mbewe, this is a disease length vector. Now, since we have multiple assets, we aren’t just concerned with the variance of each are sabai remember that it’s possible for different assets to be correlated. Thus a full characterization of the second order statistics would require the covariance matrix we’ll call the covariance matrix capital sigma. Each element in Sigma IJA is equal to the expected value of our Sibai minus Mutebi Times RSJ minus subject. As you know, this is the covariance between asset ie and asset J. If I is equal to J. This is just a variance of asset I. And as expected, Sigma is a divide matrix. OK, so what’s the next step? So far, we’ve looked at the weights which describe a portfolio, the return on each asset, the expected return on each asset and the covariance matrix. The two key statistics we are concerned with in portfolio optimization are the expected return of the portfolio and the variance of the portfolio return. We’ll call this Mississippi and Sigma Suppy squared. To understand how we might calculate these, let’s start with the simplest case where we only have two assets to consider. In this case, we only need a single way to describe the portfolio. The other way, it can just be one minus the first way. OK, so in this case, what is the return of the portfolio, the return of the portfolio? Let’s call that our sarpy is equal to W times. Our one plus one minus W times are two. Let’s make sure that this makes sense, let’s suppose that asset one returns 20 percent, an asset to returns 10 percent. Let’s suppose that I invest 100 percent of my money into asset one. In this case, my return should be 20 percent, since that’s exactly the same as investing directly into asset one. You can verify that the same thing happens if I invest everything into asset to. What if I invest 50 percent into asset one and 50 percent into asset two, in this case, I get zero point five times 20 percent plus zero point five times 10 percent, which is equal to 15 percent. So it’s a mixture of the return of the two assets, hopefully that makes sense. Let’s recognize that our support is a function of random variables, since our one and our two are random variables. This means that our sarpy is also a random variable. As discussed previously. We know that this therefore has a distribution which we can talk about in terms of its mean and its variance. What is the meaning of our sarpy? Well, that’s just the expected value of our suppy. If we take the expected value of both sides, what happens? Firstly, we can split up the addition since the expectation is a linear operator. Second, we can pull out the WS since they are not random variables. Thus the mean of our is just the weighted some of the expected return on each asset. OK, what about the variance of RP? This one is a little more tricky. It is not just the sum of the individual variances because remember that our one in our two could be correlated. The best way to figure this out is to start with the definition of variance. As you recall, the variance of our P is equal to the expected value of our P minus M.P. all squared. If we plug in what we know for RPN M.P., we just get the weighted some of our one in our two and the weighted sum of Mew One and Mutu. The trick is to group together everything with the coefficient W and everything with the coefficient one minus W, once we do that, we can expand out the square to get three different terms. The first term has the W term squared, the second term has the one minus W term squared, and the third term has two times each of the terms multiplied by each other. We recognize that each of these three terms are quantities that we already know. They are the variance of our one, the variance of our two and the covariance between our one and our two. Note that one alternative way to write this is in terms of the correlation rather than the covariance, as you recall, the correlation is equal to the covariance divided by Sigma one and Sigma two. This may be useful later, since we like to think of how related to assets are by their correlation rather than their covariance. So be aware that we may use either of these formulas to calculate the portfolio variance. There is one useful fact that we can see from our expression for the variance of a two asset portfolio, as you saw previously, when we create a portfolio from completely independent assets, we reduce our variance. But this is now the case where our one and our two may not be independent. What happens if W is a positive no less than one in this case, one minus W is also positive and therefore W times one minus W is also positive. If our one in our two are positively correlated, then the entire third term contains only positive numbers. Therefore, this increases the variance of the portfolio. What happens when r one or two are negatively correlated? In this case we decrease the variance of the portfolio. As an exercise, you may want to consider what would happen if Sigma One is equal to Sigma two and we have perfect correlation. That is, a row is equal to one. In this case, the splitting your investment between asset one, an asset to help you reduce risk. Give that a try and I’ll see you in the next lecture.

6. Describing a Portfolio (pt 2)

In this lecture, we are going to continue our discussion of how to calculate the mean and variance of a portfolio, previously we considered the simple to asset case in this lecture. We’ll move on to the more general case where we can have these assets. But before we do that, I want to show you an interesting observation that we can make about the two asset case. As you’ll see very soon, in finance, an important plot is the risk return plot. In fact, we saw this earlier in the financial basics section. This is where we plot the standard deviation on the horizontal axis in the mean, on the vertical axis. As an exercise, we can consider what kind of shape this plot would have for varying values of W. OK, so our job is to draw a graph where Moop goes on the Y axis and Sigma P goes on the X axis. Let’s begin by writing down the expressions we found from Yuppy and Sigma P squared. The next step is to essentially eliminate W. Remember, we are trying to find a Sigma Square P as a function of Muki or vice versa. We can do so by rearranging the equation from Yuppy. The first step is to expand out the one minus W term, the second step is to move Mutu to the other side and factor out the W on the right side. The third step is to isolate W. So as you can see, W is just a proportion between minus Mutu and Mbewe, one minus Mutu. The next step is to replace W. into our expression for Sigma P squared, as you can probably tell, this would get quite messy. You’re encouraged to try it yourself on paper if you’re not convinced of what I’m about to tell you. But you should be able to see it by intuition only. Basically, you can see that if we plug in our expression for a W, which is linear and moppy the highest order term we will get as M.P. squared. So we’ll have terms which have M.P. squared terms, which have moppy linear and terms, which don’t have much at all. Therefore, this is a quadratic equation in terms of moppy, we’ll have some times more squared plus some B, times more P plus some constancy. What we can see from this is that if we were to plot Sigma P squared versus moppy, we would get a parabola. However, we don’t usually like Sigma P squared. We usually just plot similar P. In this case, we would get a hyperbola, which as an exercise you can verify on your own. OK, so what happens when we have more than two assets, as you’ll see in the code lecture’s, this no longer forms a nice shape like we just arrived. However, we can still derive the mean and variance. So let’s start with the mean again. We form the random variable Arpit, which is the weighted some of the survivors. Again, we take the expected value of both sides. And due to linearity, this is just the way that some of the expected return of Icaza note that we can vectorized this operation. As you know, this sum is just the DOT product between W and the vector of expected returns MU. What about the variance, the variance requires a little more linear algebra trickery, but the final answer is simple. Let’s, as usual, start with just the definition of the portfolio variance as the expected value of RP minus moppy all squared. However, RPE is just the DOT product between W and the vector of returns are and M.P. is just the DOT product between W and the vector of expected returns. Mu. Not that I’ve actually switched around the terms in the DOT product, so I’ve written few transpose W instead of W transpose. Remember that the DOT product is commutative, so this is allowed. It might not seem clear why we’re doing this now, but it will be. The next step is to remember that the square is equal to the square magnitude, which is the same as taking the DOT product between a vector and itself. Although the thing we are squaring is actually a scalar, it’s completely OK to treat it like a one dimensional vector or a one by one matrix. Again, you might not see why we’re doing this now, but you will. So we can split up the square by saying it’s just the thing being squared dotted with itself. The next step is to factor out the WS from both terms. This gives us a W transpose on the outside left and W on the outside right. The next step is to bring the WS outside the expectation, since they are not random variables and also to simplify the transposes on the inside. What we’re left with is just the definition of the covariance matrix of our therefore the variance of the portfolio is just transpose time, sigma times W. OK, so to summarize the past few lectures, here’s what we did. First, we described all the components of a portfolio that are relevant for modern portfolio theory. These are the weights which must sum up to one, the return of each asset, their mean vector and their covariance matrix. We derive the mean and variance for the two asset case, which gives us a parabola when we plot the variance against the mean for varying values of. Not that we do not actually applied the variance, so the shape that we will see is not exactly a parabola. Next, we derive the mean and variance for the Assad case, which gives us matrixx expressions for the mean and variance of the portfolio. These will be used in all the following code lectures when we talk about mean variance optimization, the Sharpe ratio and so on.

7. Visualizing Random Portfolios and Monte Carlo Simulation (pt 1)

In this lecture, we are going to look at how to do some basic computation in order to create a risk return plot and work with portfolios in code for this lecture. We’ll be focusing on a synthetic data just to get a hang of what’s going on. Then in the next lecture, we’ll get back to working with real stocks. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So let’s start by importing pandas, numpty and matplotlib. Next, I’m going to set a random seed for non-pay, usually I don’t do this because usually I work with code that generally works the same no matter what random numbers you get. And I like my students to be able to see that variability and not be shocked by it. But for this example, there will be some code where if we don’t set the random seed, there’s a chance that your data will look very different from mine, which is not good. So setting a random seed will ensure that your results look like what you would typically expect. OK, so next, we’re going to create some parameters for a synthetic to stock portfolio. We’ll start by generating the expected returns. We’ll draw these from a normal distribution with mean zero and standard deviation zero point zero one. Next will generate some random correlation between the two stocks, we’ll draw these from a normal distribution with mean zero and standard deviation, a zero point zero one as well. Next will generate the standard deviation for the two stocks, since standard deviations must be positive, I sample two points from the standard normal and then put them through an exponential. So this will ensure that both segments are positive. Next, we’re going to create the covariance matrix. We’ll start by creating the diagonal elements, which can be accomplished by calling the Digg function on Sigma squared. As you can see, we get the squared Sigma’s on the diagonal and zero on the half diagonals. Next, we need to fill in the after signals, since we have a non-zero correlation between our two assets. We’ll call the covariance sigma one two, which is equal to Sigma one a time sigma two times row. Note that since the covariance matrix is symmetric, this same value can be assigned to CO zero one and kov one zero. All right, so next, as you know, the weights of a portfolio must come to one. One way to generate random weights that sum to one is to generate any set of random numbers and put them through the softmax, the softmax function, exponentiation, each element which makes them all positive and then divided by the sum of the exponential elements. So everything that comes out of the softmax will be positive and some to one. This is one way to generate random portfolio weights, but we’ll see others as well. OK, so next, we’re going to generate a bunch of random portfolios, calculate their expected return and risk, and then plot them in the next block. So let’s start by setting and equals 1000 portfolios. Next, let’s create empty arrays to store our returns and our risks. Next, we enter a loop that goes for insteps. Inside the loop. We generate two numbers from the standard normal and then pass them into the softmax. This gives us W which is properly normalized to sum to one. Next, we calculate the expected return. This is the mean return for each asset dotted with W.. Next, we calculate the risk. This is the square root of the variance. The variance is W multiplied by the covariance, multiplied by again. Note that in Mumbai, we work with one dimensional arrays when we want to represent vectors, so there’s no need to transpose them as we do in the math. Finally, we save the expected return and the corresponding risk into our arrays for the returns and the risks. All right, so next, we’re going to draw a scatterplot of our randomly generated portfolios with the risks on the X axis and the returns on the Y axis will set the transparency to zero point one so we can see all the points. OK, so as you can see, we get a plot that curves downward. Now you might ask, why isn’t this a nice hyperbola? Well, let’s go back up and look at what the returns actually are. So we see that the return of the first asset is about a negative zero point zero zero eight, the return for the second asset is about positive, zero point zero zero to. Notice how the return of the first asset is much larger than the second, therefore we can expect that most positively weighted portfolios with these two assets will have negative return. OK, so let’s scroll back down to our plight. As a side note, notice how the extreme ends of this plot are pretty light in terms of the dots. This is because the softmax function tends to avoid very extreme values. Therefore, the softmax function can never give you a value of zero or one, which we know is actually possible. Also recognize what the limits of this curve are at the Top End, we have the return for the second asset, about zero point zero zero two at the bottom end. We have the return for the first asset about negative zero point zero zero eight. Therefore, these correspond to when we are only invested in either the first or second asset. At every point in between is a portfolio where we have a combination of the two assets. So in the next block, we’re going to use a different way of generating our. This loop is almost the same as before, except for the part where we generate the weights. So to generate the weights, I’m going to create a single random number between zero and one. We’ll call this X, then I’m going to set W to just be a two element array containing X and one minus X. OK, so let’s run this. And let’s draw a plot. OK, so you can see that with this risk return plot, the ends are a bit darker because the random function doesn’t have any aversion to values extremely close to zero or extremely close to one. Technically, it should be uniform. Again, notice how most of the portfolio returns are negative. OK, so in the next block, we’re going to do the same simulation, but this time we’re going to allow short selling. That is to say our weights can now be negative. To do this, when I generate X, I’m going to subtract zero point five. That means that X will always be between minus zero point five and plus zero point five. W can still be an array containing X and one minus X since the constraint that the weights must come to, one still holds. So let’s run this. In this plot, the result? OK, so we can see that now we get this nice full curve with both positive and negative returns. Why is this? This is because we are now allowed to short sell on the asset with negative returns, which gives us positive returns on our portfolio. So when you short sell on an asset that drops in price, you make a profit. By the way, note that our weights are not symmetric since X is between minus zero point five and plus zero point five. What is one minus X one minus zero point five is just zero point five. So that’s the minimum value we can get. But one minus minus zero point five is one point five. Therefore, this value varies between a zero point five and one point five, unlike the first weight, which varies between minus zero point five and plus zero point five, if you want it to be more balanced, you could shuffle the weights to get a wider range. By doing so, the range of possible values would be minus zero point five, so plus zero point five. All right, so next, we’re going to look at a three asset portfolio and you’ll get to see how this plot changes. So first, we’re going to go through all the same steps to generate a mean and covariance. We’ll start by generating the means from the normal again with standard deviation zero point zero one. Next, we generate Sigma’s by exponentiation samples from the standard normal. Next, we generate our rose by drawing from the normal with mean zero and standard deviation zero point zero one notice we have three values. This is because we have the correlation between asset zero and asset one, asset zero, an asset to an asset one. An asset to. Next, we combine the Sigma’s and the Rose to generate the covariates, you might want to look at your notes to verify that what I’ve done is correct. OK, so next, we’re going to generate a random portfolios again, we have a loop that goes for one thousand steps. The main difference, of course, is in how we generate the weights in order to generate the weights. I first sample two points from the uniform distribution between minus zero point five and plus zero point five. We’ll call these x1 x to. Next, I create the weight vector W with elements X one, X two and one minus X one minus X to note that this sums to one. Let’s consider the range of values for the third value. If X one and two are both zero point five, then we get one minus one, which is zero. eFax one and X two are both minus zero point five. We get one plus one, which is two. So again, these weights are imbalanced in a simple way. To balance them would be to shuffle the order. Note that there are many ways to generate random weights, so you are encouraged to try different things. All right. So let’s run this. And let’s make our plight. All right, so what do we see? We see that this no longer forms a nice, well defined curve, instead we can see that points can now appear inside the curve. This is a consequence of having more than two assets. There is now no formula that connects the portfolio variance to the portfolio return. As a final note for this lecture, I want to note that there is a lot more stuff in this notebook that we are not going to go over in the lectures. This is because we’re going to go over them when we’re looking at actual stocks. And I didn’t see a real point in doing them twice for every topic in the section. However, it’s always nice to be able to use your algorithms on both synthetic data and real data and just doing the same exercise more than once. We’ll give you more familiarity with the material. So if you like, check out the rest of this notebook for examples on how to implement some of the other ideas we discuss later in this section.

8. Visualizing Random Portfolios and Monte Carlo Simulation (pt 2)

In this lecture, we are going to do the same exercise we did previously, but this time on real data from the S&P. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So let’s start by downloading our S&P 500 at CSFI. Next, let’s import Pendas Lumpia matplotlib. Next, let’s load in our GSV using pedigreed CSFI. Next, let’s remind ourselves what columns this particular data frame has. So we have open, high, low, close adjusted clothes, volume and name, this means that each company’s data is stacked on top of one another, since this is not the ideal format for calculating mean returns and covariance is one need to create a new data frame so that we can put each company’s data side by side. Now, let’s recall that even in this smaller data frame, we still have quite a few companies to choose from. Let’s call it a unique function to see what companies are included. OK, so this data frame contains a lot of companies for this analysis will want to keep things simple and only choose a few companies as an exercise. You might want to pick your own or even try to use them all. For us, we’re going to use Google, Facebook, access and any. Firstly, I tried to choose only one tech company to help us with diversification, and for this I chose Google. We also have Starbucks, which, as you know, is a coffee company. We have access, which is the stocktake of Recall’s, which is a department store in the United States. Lastly, we have any which is the stock ticker for Neumont, which is a gold mining company. So as you can see, all these companies are pretty much unrelated. We’ll see later whether they had positive returns or negative returns and whether they are correlated, see correlated or not correlated at all. OK, so next, I decided that for this analysis, we would be looking at the returns over a six month period. I think that seems like a reasonable amount of time. Typically, people rebalance their portfolios periodically, for example, every year, every six months or every three months. So I wanted to check roughly how many trading days there are in half a year. To do that, we can take 250 to divide divided by two. We find that the answer is one twenty six, so that’s how many samples we will be looking for, roughly speaking. Next, we’re going to get all the dates that exist in our data frame, the point of this is we want dates which are trading days that we can select from later. Now, one idea you may have had is to use the pendas date range function that we saw earlier. But as you know, this includes all days, even non-trading days. One thing you can try is if you recall, there are options to specify which days you want. For example, you might say, give me only business days or give me on the weekdays. However, I found that this does not exclude all holidays. So if you try using that, you might find that you will end up with some empty rows. As always, you’re encouraged to try that yourself, if that’s something you want to know. So in this case, what I’m going to do is take all of the values in our index and call the unique function. As you recall, our data frame currently contains multiple companies, which means the dates will be repeated. This is because we have stock prices for different companies. But on the same days next we call the sort values function, which sorts the dates. Next, we check the length of our all dates array. So it looks like we have about two thousand days of data overall. Next, let’s find our start date, since I’ve already done this, we know that it’s January two, 2014, not that it’s not January one, because that’s a holiday. Some years it might even be January three or January four. So you always have to check. Next, we do the same thing for the day since we won six months of data, which was June 30, 2014. Since this date exists in our All Dates list, we can use this date as our final day of data. If it didn’t, we would have to go down until we found a day that did exist. For example, June twenty nine hundred twenty eight. OK, so next, we’re going to assign the above two dates to variables called start and end. Next, we’re going to index all dates starting from start and going up to end plus one. Recall that the ending index in Python is exclusive. So that’s why we add one. All right, so next, we’re going to check the tape of our dates variable. OK, and we see that the variable dates is an object of date time index. And we have one hundred twenty four days in this six month period. All right, so next, we’re going to build a data frame of close prices where we put the closed price for each company side by side. This will make it easy for us to calculate the mean and covariance. So we’ll start by creating an empty data frame with the index dates, which are the dates we just found. Next, we enter a loop that goes through all the stocks in our portfolio inside the loop, we get all the rows that correspond to the dates that we’ve selected. Next, we grab all the rows which have the name we are looking for. Next, we create a new temporary data frame to store the close price for this particular stock. The important part of this is that we’re giving this temporary data frame a column name corresponding to the current stock. Next, we use the joint function to append this data to our close prices data free, as you recall, we did this exercise in the financial basics section. Next, we call the head function to ensure that our data frame is formatted as we expect. OK. And it seems to look all right. Next, let’s check whether there are any any values in our data frame. So there isn’t. Which means we can move on. Note that if you did choose stocks which had any values, either you could choose a different period or you could use forward filling in backward filling to replace that missing data. Now, recall that what we want is returns and not prices. So what we’re going to do is create a new data frame, storing just the returns, since the returns will always have one less row than the prices will index the dates starting from one. Next, we’re going to loop through all the names in our names list Inside the Loop. We’re going to calculate the return for the current stock by using the percent change function. Next, we’re going to assign the returns to our returns data frame. Note that if index the current returns using ELAC starting at the index one. Also, note that I’ve multiplied by 100. This is typical so that the numbers we see are percentages. For example, instead of seeing zero point one, you’ll see ten, which means 10 percent. This is pretty typical in these kinds of analyses. So I’ve adopted that style. Next, we call returns that head to check out new data frame. As you can see, the results are now larger, so if you see one, that means one percent. Next, we want the minute daily return over the period, so we’ll call returns that mean. Next, we want the covariance, so we’ll call returns that kov. Now, sometimes for the purposes of indexing, it’s more useful to have an umpire rather than a data frame, we can convert our covariance matrix data frame into a Nampara by calling to name PI. Next, we’re going to finally get back to our portfolio exercise, as you recall, what we did previously was create a risk return scatterplot with random portfolios. So next, we’re going to generate some random portfolios. We’ll start by setting into ten thousand the number of portfolios we’re going to generate. We set equal to the length of the mean return series. This is the number of assets. Next, we create empty arrays to store our returns and risks. Next, we enter a loop that goes for end iterations inside the loop. We’re going to generate a random wait vector. We’ll start by creating a variable called random range. I’m going to set this to one. But you can use different values, if you like, and see what effect it has. Next, I’m going to generate a random vector w by taking the uniform distribution, multiplying it by random range and subtracting random range over to. What does this give us. Well since Rande Range is one, it just gives us values between a minus zero point five and plus zero point five. In the next step, I change the last value of W to be one minus the sum of the other values. This makes it so that we meet the constraint that you must sum to one. We also shuffle W, so if there’s any particular bias towards the last value, we can at least see it for Icaza. Next, we calculate the portfolio return and risk, as you recall, the mean is the mean return of vector dotted with W.. The risk is the square root of the variance, which is dotted with sigma, dotted with W.. OK, and lastly, we assign these to our arrays of returns and risks. In the next block, we’re going to calculate the returns and risks for single asset portfolios. That means what’s my return and what’s my risk if I only invest in Google or I only invest in Starbucks? So we start by creating empty arrays called single asset returns and single asset risks. Then we enter a loop that iterates the times inside the loop. We grab the return for Asset I and the variance for asset. I note that the variance of asset I is just the covariance arrow. I call my next. We assign the current return and risk to our single asset returns and risks arrays. Next, we cross scatterplot of all the returns and risks that we just found for the random portfolios will set Alpha equal to zero point one so that the dots are transparent for the single asset portfolios. We will not use transparency and we’ll also color these red to distinguish them from the random portfolios. All right, so what do we see? So, again, we see this sort of bullet shape, by the way, this is called the Markowitz bullet. Inside the bullet, we can see all of our individual assets, which are colored red. So there are a few interesting conclusions we can draw from this. This graph shows that by diversifying our portfolio, we can reduce our risk but receive the same return. For example, take the red dot with the highest return. This has lots of risk, over two percent. However, we can see that it’s possible with the correct setting of the weights to achieve the same return, but with much lower risk. We can see this by looking horizontally on our plot. Alternatively, we can see that if you don’t mind taking on a level of risk as high as the highest single asset, we can still achieve higher return for this given level of risk. We can see this by looking vertically on our plot. Furthermore, we can see that by taking short positions, we can achieve returns which are higher than the highest return of any single asset. The last thing I want to point out about this graph is notice the very odd shape. You’ll see that if this were a more perfect curve, it would go a bit higher, whereas this seems to flatten out at the top. This is due to the limits that we’ve set on our weights. You’ll see that if you change those limits, you will get a different shape for the possible portfolios. One downside to portfolio optimization is that in order to achieve those portfolios, we would have to take some pretty extreme positions. For example, you might have to short sell or borrow one hundred percent or more of your wealth. This is actually pretty dangerous because as we’ll discuss later, short selling is very risky. And on top of that, it’s common to have to pay interest on your short positions. In other words, borrowing money is not free. And furthermore, it can be quite dangerous just as it is in the real world.

9. Maximum and Minimum Portfolio Return

In this lecture, we are going to start discussing ways to characterize what we know about our portfolio. Now, most people usually just skip to a mean variance optimization, but I think there are more fundamental questions to ask. For example, given a set of assets, what is the highest return I can achieve? What is the lowest return I can achieve? In this lecture, we will answer these questions. Let’s start by considering the maximum return on a portfolio of assets, as you know, the return on our portfolio given away Vector W and the expected returns, AMU is just the DOT product of W and HMU. So you might say just use calculus to find the maximum. To be clear, we’re trying to find the maximum of Mbewe transpose W with respect to W, that is what value of W will maximize this function. However, this doesn’t work y well for several reasons. First, the maximum of this function is infinity. You can imagine that this is like a plane or in one dimension it would be a line. Lines can go on forever. So the maximum is infinity. Second, this disregards one important fact, which is that the sum of the weights must equal to one. Otherwise, it’s not a valid portfolio. In actuality, this is a different kind of optimization problem, instead of just having a function to maximize or minimize. We also have constraints in general. We call these constrained optimization problems. We can state our problem as follows. We want to maximize Mbewe transpose W with respect to MU, this is subject to the constraint that the one vector dotted with W is equal to one. Note that this is just a fancy way of saying that the sum of all the elements of W must equal one in optimization. We like to represent every equality and inequality in terms of vectors and matrices, since that’s eventually what we need to put into the code. So the one vector is just a vector containing all ones. You can verify on paper that this is the same thing as saying that the sum of all the Savi’s must equal one. Also, since there’s no way for me to Baldivis equations, I’ve subscripts that the one vector with a D to make it more obvious that it is a vector of sized containing all one’s. Now, if we stare very hard at our new problem, we will notice one unfortunate fact it still has a maximum of infinity to see this. Suppose we just have to assess with returns at minus zero point one and plus zero point one. Well, we can obtain a positive return by shorting the first asset and going along on the second asset. For example, let’s say we put minus 100 percent into asset one and two hundred percent into asset two. In this case, our return will be minus one at times minus zero point one plus two times zero point one, which is equal to zero point three or 30 percent. But why can’t we be more extreme? How about minus 200 percent in asset one and 300 percent in asset to this still sums to one. In this case, our return will be minus two times minus zero point one plus three times zero point one, which is equal to zero point five or 50 percent. Obviously we can just keep increasing the amount that we short on the first asset and then we’ll have more to invest in the second asset, which will keep increasing our return. So this is not good. In reality, we often have additional constraints on the weights of our portfolio. For example, we might say don’t invest more than 30 percent into any particular asset. This will keep your portfolio diversified. We might also say don’t short any more than 10 percent on any particular asset. In many cases, shorting isn’t allowed, period. For example, your boss may simply tell, you know, short selling. Remember that short selling is risky. One reason why this is, is because you can theoretically have an infinite amount of losses. Why is this? Well, if you purchase some shares of a stock that is you take a long position and that stock goes down to zero, then you will lose your original investment. So if you go long, then the maximum you can lose is your entire investment. But when you short sell, remember that you’re betting on the stock going down. If the stock goes up, you lose money. And since there is no limit to how high a stock can go, there is no limit to the amount of money that you can lose if you short sell. That is to say you can lose more than just your entire investment. All right. So we would usually like to have additional constraints on the weights, not just that they should come to one. So here’s an example of a constrained optimization problem with more constraints on the weights. This time we still want to maximize, muite, transpose w with respect to HMU, this is subject to the constraint that the one vector dotted with W equals one and that each W Sibai is greater than or equal to zero. And by the way, we have a special name for this particular kind of optimization problem. We call them linear programs or LP’s for short. The reason these are called linear programs is because they are optimization problems where the objective is linear and the constraints are also linear. If either of these were non-linear, then it would not be called a linear program. We will see a few examples of that later in this section. From here, it’s easy to find the minimum return as well. We simply switch the max to women and the constraints remain as they were before. OK, so now that you know what a linear program is, how do we actually solve them? Luckily, there are libraries in many programming languages to help us do this in Python. We use the Lynn Praag function, which is included in ZIPPI. So the Lehnberg function works like this. Firstly, by default, the Lehnberg function does minimization and not maximization in zippi the function that we want to minimize is C transpose X with respect to the Vector X.. So if you want to maximize, you just pass in the negative. Obviously this is subject to several constraints. First, we have inequality constraints. We say that Ayyoub Times X must be less than or equal to be U.B.. Second, we have equality constraints. We say that ague times X must be equal to Bhiku. Finally, we have bounds. We can say that X must be bounded below by L and bounded above by U. Note that just because Sipi gives you these options, it does not mean you have to use them. For example, you may not have any upper bound on your variables, so you can just leave this argument as the default, which is known. You may not have any inequality constraints so you can leave those as known. In other words, you only have to specify the parts that you actually need. Now, using sidepiece definition of a linear program, it might not be clear how exactly we can implement our problem. We have that the sum of the weights must equal to one. But this constraint doesn’t seem to be compatible with what Sipi gives us. In reality, we just have to stretch our imagination to suppose that we have three axes x1 x to an X three people need to sum to one. Let’s say we have a Matrix IQ equal to a one by three matrix of ones. Finally, let’s suppose that we have BQ, a one by one matrix containing just a single one. We can see that by using the constraint equal times X equals BQ. We get exactly that. The sum of the Xs must equal to one. In fact, if you look at a more typical definition of a linear program like what you would find on Wikipedia, it would look like this. As you can see, there is only an inequality constraint and no equality constraint. In addition, there are no bounds. There is a constraint that X must be greater than or equal to zero by default. In general, there are some tricks you can apply to convert inequality constraints into equality constraints. There are also other tricks you can apply to convert upper and lower bounds into inequality constraints. So everything we can do in ZIPPI can be converted into the format that you see here. This is typically something that you would learn in a course that focuses on linear programming, convex optimization operations, research or management science. Luckily, Zippi makes things a little easier for us by giving us more flexibility in how to represent our constraints.

5. VIP Algorithmic Trading

1. Algorithmic Trading Section Introduction

In this lecture, we are going to begin the next section of the course, which is on algorithmic trading. So just to tie this into what we have done so far, the last section was all about how to allocate a portfolio. The point of this is to select from multiple stocks such that we maximize, return and minimize risk. At that point, we were already beyond asking the question, how do we forecast a stock price so that we can pick the best stock and make lots of money in this section? We continue along this theme, but on a different axis. The previous section was all about which stocks to buy and in what proportions. This section is all about when to buy and also when to sell. So instead of considering assets, we now consider time. So let’s do a short outline of this section, we’re going to start by introducing a traditional approach called a trend following, you may be surprised to see that this only depends on the simple moving average, which is a concept we discussed much earlier in the course. At that time, it was mostly just a stepping stone to the exponential moving average. But you’ll see that this simple concept can be applied to trading as well. After this, we will take a machine learning based approach where we attempt to use a machine learning model to predict whether tomorrow’s return will be positive or negative. If tomorrow’s return will be positive. Then we would want to buy. And if we predict that it will be negative, then we would want to sell. Of course, if we could do this perfectly, then we would never lose any money, even on stocks that trend downward. The ultimate goal of our study in algorithmic trading is to get to keep learning, keep learning as an algorithm from the field of reinforcement learning, reinforcement learning is kind of a tough subject. So it will not be discussed in this section after the. We will go through a brief introduction to Q Learning, at which point we will be ready to apply it to algorithmic trading. Once that’s complete, we will return it to algorithmic trading and apply the things we learned. We’ll start by implementing the trend following strategy, but using a reinforcement learning setup. This will help us understand the flow and design of a reinforcement learning program. We’ll also be able to sanity check our code from this section to ensure that doing things both ways will give us the same answer. After that, we will be ready to implement key learning.

2. Trend-Following Strategy

In this lecture, we are going to look at our first trading algorithm, which is called a trend following strategy. This is a traditional trading technique which is not based on machine learning. If you recall earlier in the course, we talked about simple moving averages. At the time, they probably didn’t seem that useful in this lecture. We will see how the simple moving average can be applied to build a trading strategy. OK, so the best way to understand the trend following strategy is to look at it visually, the basic idea is this we’re actually going to create two simple moving averages out of a stock’s price. One will be called the fast moving average and one will be called the slow moving average. Intuitively, we know that if we include more terms in the moving average, then it will be slower to adjust. That’s simply because each individual value in the moving average will have less influence on the result. Therefore, if we had one moving average looking at the past 30 days, it would be slower than a moving average that looks at only the past 10 days. As you recall, slow moving averages are smoother while fast moving averages are more jagged and react more quickly to the input. OK, so that should just be a review of what we learned in a time series analysis. All right, so the basic algorithm works like this. I’ll start by telling you what it is and then we’ll try to break it down to make sure it makes sense. The key is to look at the points with a fast moving average crossing, the slow moving average, whenever we see the fast moving average across the slow moving average from below. We consider that a signal to buy whenever we see the fast moving average across the slow moving average from above. We consider that a signal to sell in between these values will assume that we hold the position we were in previously. For example, if your previous action was to buy, then that just means you’re holding stock and hopefully while you’re holding it, it rises in value. OK, so crossing from below means by and crossing from above means sell. The question now is, why does this strategy make sense? Let’s consider the first case where we buy because of the fast moving average crosses, the slow moving average from below. This means that the stock’s price is trending upward. The fast moving average captures this upward trend, but the slow moving average is taking a bit more time to react. Of course, this is predicated on the hope that this upward trend will continue. Now, consider the case where we sell, we sell when we see the fast moving average across the slow moving average from above. This means that the stock is trending downward. The fast moving average captures this downward trend, but the slow moving average is taking a bit more time to react. Again, this is predicated on the hope that this downward trend will continue. Let’s not talk about some assumptions and restrictions we’ve just made, firstly, in the following lectures, we will assume that you will not shorts out. That is, either you will hold stock or you will hold cash. So you won’t be able to make money when the stock price goes down. Second, and this one’s pretty obvious, we are only working with a single stock or a single asset such as the S&P. Of course, if you want it to run this algorithm on two assets, you could do that, but they would be completely independent. Third, we assume that whenever we buy in, whenever we sell, we take this action using the entirety of our wealth. So suppose we’re holding cash and we’re going to buy. That means we’re going to buy every share that we can. And after that, we will hold zero cash. In reality, this is not exactly possible because traditionally you can only buy whole shares, not fractional shares. Furthermore, when we sell, we’re going to sell every single share that we own. So after selling, we will own zero shares of the asset under consideration. Now, it’s important for us in the upcoming code lectures to get the timing right and the rest of this lecture, I’m going to outline roughly what the code will look like. So you will have a chance to consider it as a thought experiment before we actually run the code. It will seem kind of strange because we’re talking about trading, which implies that we’re doing some action like buying or selling some shares of a stock. But in reality, all we’re going to do is manipulate some values in a data frame. So in practice, it may not seem like trading, but keep in mind what it represents. People call this paper trading because you’re simulating what would happen in a live environment. In fact, if this were one hundred years ago and we did not have computers, you would actually do this on paper. So it will seem strange, but it does actually simulate trading in later lectures. When we discuss reinforcement learning, we’ll actually build an entire program that separates the trading logic from the market so that they exist as two separate entities in the code. This might make it more clear what we are actually simulating and it may appear more natural. Furthermore, this will help you kind of sanity check everything we’re about to do. Where we’re going to do is implement the trend following strategy once in the next lecture and then once again using a reinforcement learning paradigm. And you will be able to verify that both methods give you the same answer. OK, so anyway, I said we want to get the timing right. You’ll see why this is important as we manipulate our data frame, because what we will do will seem quite odd. OK, so first, let’s assume that we calculate the fast and slow, simple moving averages. These are just two new columns in our data frame. Let’s assume that we know how to use these to generate, buy and sell commands. So we’ve done that. And now we have a new column where it says Buy, sell or nothing. OK, and let’s recall that our table also includes the return for each day. Now, you might assume that in the row where it says buy, we will get the return a calculator for that day. In fact, this is not the case. So let’s zoom into how the timing will work. As you know, in order to calculate the returns, the moving averages and so forth, we will use close prices. Now, this is just an approximation. Let’s assume that we can calculate the simple moving average using the price just before the market closes so that if we do decide to buy, we will buy at the close price for that day. Remember, you cannot buy anything after the market is closed, just as you cannot order food from your favorite restaurant after it’s closed. Again, this is just a very rough assumption, but let’s assume that it holds. So let’s say we purchased some shares at the close price at the end of some day and say they cost you one hundred dollars. We’ll call this day time at T minus one. The next day the price goes up to one hundred one dollars. This is called the time t note that the return, which is one hundred one divided by one hundred minus one, is the return of time T, not time T minus one. So the return that you get corresponds to the day after you bought your shares. So what’s the consequence of this? Basically, it just means you need to make sure that your returns line up properly in your data frame. One way to implement this, which will make things easier for us, is to simply shift the returns themselves backwards by one step. This is because in Panda’s operations tend to happen along roads, and so doing operations across different rows is more challenging by placing the next day’s return along each row. We can more easily calculate the returns from our algorithm. So instead of discussing this theoretically, let’s look at how we’re going to do this in code. OK, so at this point all we have, our two moving average is the next step is to determine where the fast moving average is greater than the slow moving average. We’ll call this the signal column. The next step is to shift that column forwards into a new column called Previous Signal because pendas operations happen. Realize we can now check by using these two columns when the fast moving average crosses the slow moving average, either from above or below. If the previous signal is false and the current signal is true. That means across from below. If the previous signal is true and the current signal is false, that means it crossed from above. OK, and so each time this happens, we’ll say we want to buy or sell. To do this will create two new columns, buy and sell, and these will contain booleans. The buy column will be true whenever we see the crossing from below and the cell column will be true whenever we see the crossing from above. Please note that this table is not intuitive, so you may want to go through this slowly to ensure that this logic makes sense. Next, we’re going to have a column called Is Invested. This column will tell us whether or not the value of our wealth will be affected by the returns. If we are invested, then our wealth will change by that day’s return. If we are not invested, then our wealth will stay constant since we are just holding cash. So in the code, we are going to use the pandas apply function to assign this column. But in this lecture, I’m going to show you a basic for loop, which is hopefully more intuitive. So we’re going to start by creating a global variable called is invested and will always start off not being invested. So is invested as follows. Next, we’re going to loop through each row of the data frame inside the loop. We’re going to check two cases. If we are invested and the command is the cell then is invested, becomes false. If we are not invested and the command is to buy, then is invested, becomes true. Otherwise we do nothing and is invested, remains, whatever it is. At the end of the loop we assign the variable is invested to the column is invested. Now one important thing to notice about this column is invested is that just as we discussed earlier, it’s off by a day. Why is that? This is because if we buy at the end of today, we are not technically invested and earning or losing money until the next day when we’ve calculated the return. However, having this column like this is convenient, in my opinion. Note that there are many other ways to implement this algorithm. So if you don’t like this idea, you’re always encouraged to design your own version of the code. In fact, in my pure machine learning courses, that’s often my initial instruction. Machine learning students should be able to take an algorithm description based purely on the theory and implement that theory in code without any assistance. Other people’s code should be completely irrelevant. So you may want to take that strategy here. Remember that I’m not telling you how you should implement this. I’m simply sharing one possible example with you. You are strongly encouraged to write your own, whichever way is most intuitive for you. OK, so the final step, once we know whether or not we are invested, is to calculate the return of our algorithm in Python, since true equals one and false equals zero, we can simply multiply that is invested column by the shifted return column and then sum up the results. Remember that we shifted the returns back by one day, so we’re actually getting the returns for the next day. That’s why there’s no need to shift that is invested. So why does this make sense? Well, if we are invested, then we will get the return for the day we are invested. That’s equivalent to multiplying by one. If we are not invested, our return is zero since we are just holding cash. So that’s equivalent to multiplying by zero. Oh, and by the way, we’ll be working with log returns. So the total cumulative return will just be the sum of the individual daily log returns.

3. Trend-Following Strategy in Code (pt 1)

OK, so in this lecture, we are going to implement the trend following strategy in code, this lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend, is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. Let’s start by downloading the spy CSFI, since the strategy uses only the historical prices of the asset under consideration. All we need is just a single time series of close prices. Next, we’re going to import pendas, numpty and matplotlib. Next, we’re going to use prediabetes, ESV to read in our CSFI. Next, we’re going to call DFG head to remind ourselves what is inside our data frame. In the next block, we’re going to find the log of the closed prices and then take the difference to get the log returns. Next, we’re going to do a deft had again to remind ourselves that when we calculate the return, the first value is missing. Next, we’re going to shift the log returns backwards so that the rows line up with the buy and sell commands that we will create later, note that this will get rid of the missing value that we just saw. So if we do that again, we can see that the first value is no longer missing and everything has been shifted down. This does mean, however, that the last value is now missing. Next, we’re going to calculate the rolling meme with sample size 30 and sample size 10. Now, you might ask, how did I choose these numbers? The answer is currently we are just looking at a random set of numbers. Later in this lecture, we will try to choose these values intelligently. Next, we’re going to plot the close price along with the moving averages. As you can see, it’s not easy to tell where the crossover points are, since there are too many data points. In the next block will limit the plot to just the first three hundred points. Try to look at this and guess where the algorithm will decide to buy and sell. Next, we’re going to calculate where the fast Esmé is larger than the slow Esmay using the NPR where function. This will return to one where the argument is true and zero. Otherwise, conceptually, you can think of this as like a vectorized if statement. Next, I’m going to print out a portion of the data frame as it currently exists. Obviously, I chose these points because they have a mixture of ones and zeros under the signal color. So check this out for yourself and make sure that it’s doing what we think it’s doing. That is when FASTA Sam is greater than slow Esmay, we should have a one. And if it’s not, then we should have a zero. Next, we’re going to shift the signal column forwards so that we can look at the previous signal on the same row that makes the next lines easier, where we determine where to buy and sell. Recall that we want to buy when the fast assume overtakes the slow s.b. In other words, the fast Asabi was previously less than the slow Esmay and it became greater. The opposite is true when we want to sell. OK, so look at this code and confirm to yourself that it does what I’ve just described. Next, I’m going to print out the same portion of our data frame that we were looking at previously, recall that the signal went from one to zero. This means that faster Samay was greater than slow ESMAY and then it became a less than slow Esmay. This means that Faster Samay was greater than slow Esmay and then it became less than slow Esmay. Therefore, we should find a self-command here. And in fact, that’s exactly what we find. Next, we’re going to write a function that will determine for each step whether or not we are invested, we’ll start by initializing a global variable is invested to false. Next, we’re going to write a function that will be called on each row of the data frame. Using the pen does apply function. So this function takes in one argument, which is the row of a data frame. This is why we need a global variable since we can only look at one row at a time. It’s an easy way to keep track of the state of being invested or not invested. Inside the function we declare is invested as a global variable so that we can modify it within the function. Next, we check the various cases where we need to reassign is invested, if we are already invested and the command is to sell, then is invested, becomes false. If we are not invested and the command is to buy, then is invested, becomes true. Otherwise is invested, does not change. And it remains. Whatever it is, this function returns, is invested, which means that when we call DOT apply, this will return to a series containing billions. Next we call the DOT apply passing in the function we just created. We assign the result to a new column called Is Invested. Next, we calculate the return of our algorithm by multiplying the is invested column by the log return column. Next, we calculate the cumulative log return of our algorithm by calling the sum function on the algo return column. We get about 30 percent, which seems good, but let’s check the buy and hold return. This just means that we invested on day one and never sold. Therefore, the cumulative log return is just the sum of the log return column. When we do this, we see that the result is seventy eight percent. Therefore, the trend following strategy performs worse than simply buying and holding. Next, let’s calculate the standard deviation and Sharpe ratio of our algorithms returns, we’ll assume that the risk free rate is zero for simplicity. OK, so we get about zero point zero zero six and zero point zero two. Next, let’s do the same for the buy and hold strategy, which is just the return call. OK, so we get about zero point zero zero nine and zero point zero three six six. So although our algorithm has lower risk, it also has a lower Sharpe ratio compared to buy and hold. Buy and hold is superior.

4. Trend-Following Strategy in Code (pt 2)

Now, you might wonder what would happen if we tried to optimize the number of values we use in the fast and slow Esmé in the next part of this lecture? That’s exactly what we’re going to do. The basic idea is we’re going to split the data into a train set and a test set. We’re going to use the train set to select the fastest and the slowest, and then we’ll evaluate it on both the train and test set. In other words, we will choose the the same and slow smarter that give us the best return on the train set. Only in machine learning parlance. You might think of the test as a validation said, OK, so this function, it takes in three arguments. The full data frame of close price is the fast parameter and the slow parameter inside the function we’re going to again declare is invested as a global variable. Next, we’re going to calculate the fastest time and the slower, some using the values of fast and slow that we passed in. Next, we’re going to repeat all the steps we did above. First we find a where fast assembly is bigger than slow Esmay. We assign either one or zero based on the result. Next, we assign the previous signal column by shifting the signal column by one. Next, we set the buy and sell commands using the same logic we had before. Notice how we do all that work before we split the data into train and test. So why do we do it this way? Remember that when we calculate the ESMAY, the first few values will be missing until we have enough values to actually calculate decimals. So if you want some ten, then for example, the third value is going to be missing because you can’t calculate some 10 from only three values. Now, the result of this is that in the whole data frame, only the first few values are missing. That means only the train set has missing values, but not the test set. If we had split the data into training tests first, then you would have missing values in the test set as well. But that doesn’t really make sense, since in the real world, you would have historical data when you begin testing your strategy anyway. If that doesn’t make sense to you, you can safely ignore what I’m saying. I only mention this since it’s a thought that I had when I was creating the script. Basically, the function would take in only the train set or only the test set and apply the algorithm on each of them individually to this function. On the other hand, it takes in the whole data frame and splits it into training test and then gives you back both the train return and the test return together again. If that doesn’t make sense to you, it’s not necessary to understand. All right. So next, we’re going to assign that is invested column. Since we always start out not invested, we have to assign the global variable is invested to false. Next, we call train dot apply. Next we multiply is invested by the log return. Next we do the same process on the tests. Finally, we return both the train score and the test score. Now, you’ll notice I’m doing something weird here, which is that I’m not including the last value in the sum. This will make sense right now, but it will when we look at the reinforcement learning version of this script, intuitively, you can just think of the test set. The test set has no final return because we shifted them earlier, if you recall. Therefore, on the test set, this actually has no effect. But on the train set it does. So another reason is just for uniformity. This has more to do with the fact that there are limitations of working with data frames and not having data that goes on forever. So this is just some housekeeping we’re going to do so that the results we get now are in line with what we will get later. OK, so next, we’re going to test that our function works, we use the same fast and slow parameters that we used earlier so that we can sanity check the result. OK, so we get about zero point two one nine zero point zero nine, which some up to 30 percent. So that makes sense. Next, we’re going to do a grid search to find the best fast and slow values. We’ll start by setting best fast and slow to none. Well, initialise best score to minus infinity. So there any score we get can beat it. Next, we loop through the fast and slow values to try to note that these are pretty arbitrary. They’re kind of like hyper hyper parameters. Feel free to test your own values if you don’t like these. Again, I’m just showing you the method, not telling you what the best values are. OK, so inside the loop, we’re going to call a trend following function with the current values of fast and slow. Note that we ignore the second return value because we will only use the train score to evaluate the parameters. If the train score is greater than the current best score, then we save the score along with the current values of fast and slow. Finally, when this loop is complete, we call the function one last time on the values of Best Fast and Beslow that we found. OK, so let’s run this. OK, so we see that for the train score, we get only forty three percent and for the test score we get only nine percent. Of course, these values only make sense in the context of the baseline, which is buy and hold. So next, let’s split the data into train and test and make copies for later. Next, will some the log return column for the train said again, we come only to the second last value. So the result is about 60 percent, again, to buy and hold wins. Next, let’s look at the test set again, something to only the second last value has no effect since the last value is actually missing. And again, buy and hold wins, we get about 19 percent. Now, it might be worth checking the Sharpe ratio as well, just to see if our strategy has any redeeming qualities. So in this block, we’re going to print the Sharpe ratio of the algo log return as well as the log return. We see that the Sharpe ratio of the algorithm is slightly higher, so not being invested does at least reduce variance. Next, let’s do the same thing for the TSA. Unfortunately for the test said, buy and hold wins again. OK, so the last thing we’re going to do in the script is plot our wealth over time since we worked with log returns and not regular returns, we have to do this carefully. First, we’re going to calculate the cumulative log return for each time step by calling the come some function. We’re also going to shift this forwards since, as you recall, the returns are currently shifted backwards. Next, we’re going to calculate our cumulative wealth on each timestep. This is equal to the initial price multiplied by the exponential of the cumulative log return. Now, you might ask, how do we get this? OK, let’s suppose we want to calculate the log price at time t from the log price at time zero. We find that P1 is equal to zero plus R one, P two equal to P1 plus R2, but that is equal to zero plus R one plus our two. We can keep repeating this pattern until we find that time T is equal to time zero. Plus the sum of the log returns up to time t. Of course, if we just take the exponential of both sides, we recover the price at time t the price at time zero and the exponential of the sum of log returns. OK, and the last step is to simply call the plot function on the relevant columns. So what do we get? Well, it may be a bit hard to see, but you’re encouraged to zoom in on your own computer, basically look for all the areas where we have a straight line on our cumulative wealth. These should start at the cell points and end at the by points. Unfortunately, it seems like our algorithm reacts too slowly, so we end up investing after the stock rises and selling after the stock drops. One fact we know is that the exponential moving average doesn’t lag as much as the simple moving average, so you may want to give that a try as well.

5. Machine Learning-Based Trading Strategy

In this lecture, we are going to continue our discussion of algorithmic trading, the previous trend following strategy can be thought of as having hyper parameters, but not parameters. There was no learning and no statistical model was involved in the strategy. In this lecture, we will be moving to machine learning based strategies. We’ll see how building a predictive model of future stock returns may be able to help us trade better. Now, you may recall one lesson we learned earlier in this course, it was that predicting stock returns is pretty hard, especially predicting multiple days into the future. But what if we only need to predict one day into the future? This, in comparison, may be an easier task in our current scenario. This approach is plausible because just like the trend following strategy, we are going to update our position on a daily basis. That means we can use all the data up to the current time step and only predict the return for the next timestep. Our decision to buy, sell or do nothing is updated every single day. Therefore, we only have to worry about predicting one day into the future. OK, so that being said, I think our problem becomes pretty simple. We’re going to build a predictive model to predict tomorrow’s stock return and buy, sell or do nothing accordingly. So if we predict that it will go up, then we buy. And if we predict that it will go down and we sell. Note that instead of the auto regressive approach we discussed in the times of recession, we are now going to take a different approach. As you know, the S&P 500 contains approximately five hundred stocks. It is plausible that the previous returns on these 500 stocks could predict the next return of the total index. Note, however, that this is not guaranteed. It is just an idea. In actuality, I would remind you that one of the major themes of this course is it’s not about the model you use, but the data. If you use non predictive data and it doesn’t matter how good your model is, it just won’t work. But the data you could use really consists of anything in the world. And so you really have to consider that when building these models. Now, obviously, using all the data in the world is not feasible for a course like this. And the possible combinations grow exponentially as we consider more sources of data. Even with 500 stocks, the number of combinations we might try is in feasibly large. Therefore, what I’m going to do is pick a small subset of stocks now, you might ask, did I do anything special to pick these stocks? The answer is no. I simply chose some companies from the S&P 500, which have the largest market cap. Another question you might have is why not just use all 500 companies and let the model do the work? The answer is that this will overfit. Now, the reason why is beyond the scope of this course. But if you want to learn more, you’re encouraged to check out my in-depth series where I go through the math and theory behind algorithms such as linear regression. Now, obviously that being a course all by itself, we don’t have time to discuss that here. However, I will remind you of my motto. Machine learning is experimentation, not philosophy. If you think something will work, you shouldn’t try to guess what the outcome might be. Simply trying it and observing the result is always a superior choice. Another question you might have is, why am I using stock returns and not stock prices to predict the S&P 500? The answer is, again, outside the scope of this course. But machine learning models and statistical models are generally not good at extrapolation. As you know, stock prices in the market are generally going up. If you train your model on prices in the range one hundred dollars to two hundred dollars, your model might learn what to do for that range. But if it goes up to three hundred dollars in the test set, your model has never seen that before and it doesn’t know what to do returns. However, as we learned, are more or less stationary. Therefore they are better candidates for input features. Now some of you who have done machine learning before might ask, why can’t I just normalize the data? For example, put it into the range at zero to one. In fact, this would not work. And I would encourage you to think about why you will still have the same problem. OK, so now that we’ve done all the groundwork, let’s see what our data frame paper trading code will look like. Suppose we have a data frame, a full of stock prices in each column for various assets in the S&P, as well as the price for the S&P itself. From this, we can create a new data frame containing only the corresponding returns. Note that because we’re calculating returns or log returns, the first row for all of these columns will be no. Therefore, no inputs or outputs will exist for the first row, unlike the trend following strategy. The next step is again to shift to the S&P returns backwards by one, as you recall, data frames in arrays are easier to work with when everything we need to compute on is in the same row. Remember that we are using the returns of the input stocks to predict the next day’s return of the S&P. Therefore, it’s easier to have all the inputs and corresponding outputs in the same row. Once we have this, it’s easy to split our data into train and test. We can simply make the last one thousand rows, the test set and everything before that, the train said. Next, what we’re going to do is take all the columns from the train set that belong to the input variables and call that X train, we’ll call the S&P returns column. Why train then? All we need to do is create a machine learning model like linear regression and fit it on X train and Y train. Note that you’ve seen this before in this course. Now, you might wonder what will happen if we use more complex models or different kinds of models. These will be discussed in a future lecture. So just hold that thought for now. OK, so the next step will be to use our model to make predictions, not that we don’t actually need to store the predictions themselves, just whether they are greater than or less than zero if our prediction is greater than zero. We will buy or just remain invested. If our prediction is less than zero, we will sell or just remain on invested. So at each row of the data frame, we will know whether we are invested or uninvested based on the sign of the prediction. Once we know whether we are invested or not, we can do the same thing we did previously where we multiplied the invested column by the returns, we can assign that to a new column called Algo Return. The total cumulative log return is then just the sum of this column. OK, so that’s the algorithm for building a machine learning based trading strategy.

6. Machine Learning-Based Trading Strategy in Code

In this lecture, we are going to look at machine learning based trading in code. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So let’s start by downloading our data said. Unlike the trend following script, we will need more than just the speed prices, as you recall, we’ll be using past returns of the components of spy to predict spy. Again, I want to mention that the number of things you could use to predict a spy are exponentially large. In fact, even with just this data set, there are a lot of things you could try. OK, so next, let’s import pandas, numpty and matplotlib. Next, let’s read a Naqvi using Pedigree’s CSFI. Next, let’s call DPG head to remind ourselves what’s inside our data frame. As you recall, we have the clothes prices of various stocks from the S&P lined up by day. Next, we’re going to drop any rows in which all the values are missing, so we call drop in with access equals zero and we say how equals all? Next, let’s check the shape to see how many points we have. Next, we’re going to remove any columns with missing values. This is just so we don’t have to bother with forward filling or backward filling to do this. We call drop in with axis equals one and we’ll see how it goes. Any. Next, when we check the shape again, we see that about 50 stocks have been removed. Next, we can call is N.A as a sanity check to make sure that there are no more missing values. Next, we’re going to create a new data frame containing only returns, this will have the same structure as our existing data frame, but it’ll contain returns instead of prices. So first, we start by creating an empty data frame. We’ll call this D.F. Returns. Next, we live through each name NDRF columns Inside the Loop will assign the log return to D.F. Returns under the same column name. Next, let’s call the head function to remind ourselves what our data frame will look like. As you can see, the first row is on No. Next, we’re going to shift to the spy returns back by one step, as you recall. This is so that we have all the inputs and targets along the same row. If we do a detail, we should see then. Now the last value is missing. OK, so the next step is to split our data into train and test, since the first row has missing values. We’ll skip the first row for the train set since the last row has missing values. We’ll skip the last row for the test set. OK, so next, we’re going to select which stocks we want to use as our input features. As mentioned in the Theory lecture, I simply chose a few stocks that have the highest market cap. Alternatively, you could try using all the columns except obviously S.P.I itself. OK, so we’ll call this list X calls. Next, we’re going to create our X’s and Ys for building and testing our machine learning models. X is the data frame index by X calls. And why is the data frame indexed by SPI? Next, the let’s call extranet head as a sanity check to make sure everything looks OK. All right, so everything looks fine. Next, let’s call White Head to make sure everything looks OK there to. All right, so everything looks fine. Next, let’s create our model, as mentioned, we’ll start by using linear regression, we call Model Duffett to fit the model on the train set. Then we call model that score on both the train and test set to evaluate how close our model gets to predicting the target. As you can see, the scores are quite low, a perfect score would be one, and predicting the average value would be zero. You can think of that as like the naive prediction. So perfect gives you one and naive gives you zero. For the transfer, we are only slightly above the naive prediction for the test set since we get a negative value. We are actually doing worse than the naive prediction. Nonetheless, this won’t stop us from just seeing how the model performs. OK, so recall that we don’t actually care about the value of the prediction, just whether it’s positive or negative. Recall that if it’s positive, then we want to buy. And if it’s negative, then we want to sell so we can obtain predictions from the model by calling model does predict this will give us a P train and P test. Next, we can check the accuracy of Cranham P test by taking the sign. So when we call the sine function, this converts the array into plus one, if the argument is positive and minus one if the argument is negative, if we do this for both the predictions and targets, we get a raise of plus ones, a minus ones. Then we check where they are equal, which will give us an array of booleans. Finally, we take the mean of the boolean array to get the classification accuracy. As you can see from both the train and test that we perform slightly higher than chance since there are only two possible outcomes, the expected result of making random predictions is 50 percent. Now, one subtlety about the sine function is that if the argument is zero, we will get zero instead of plus or minus one. Therefore, it’s possible to have three values instead of two. So in this next block, I’ve casted P train and P test to a set to see how many unique values we have. Luckily, they do not contain any Xeros. OK, so the next step is to create a new column in our data frame called Position, which will tell us whether or not we are invested for the train, the seats we assign, the bullion array P train greater than zero. That is, if P train is greater than zero, will assign true. Otherwise we’ll assign false. Next, we do the same thing for the test indices and P test. Next, we multiply the position column by the spy column to get the Al Gore return. Next, we sum up all the values corresponding to the train set to see how well our model performs. So we get about fifty five percent log return. Next, we sum up all the values corresponding to the TSA. We get about 30 percent log return. Next, let’s compare this against buy and hold, so far, the train set, that’s why train got some. So we get 59 percent longer term, which is better than the machine learning approach. For the tests that we use, we tested some. We get 19 percent, which is worse than the machine learning approach, so the machine learning approach outperforms, buy and hold on the test set for this time period and this model.

7. Classification-Based Trading Strategy in Code

OK, so this lecture is a continuation of our previous notebook where we will train a model to predict whether tomorrow’s return will be positive or negative. Previously, we used linear regression, which is a model that predicts a real value target. However, you may have noticed that we never actually use the value of the model predictions, only the sign. So you may recognize this as a classification problem. In this lecture, we will train a classifier directly and see how well it performs. We’ll use a model called Logistic Regression, which is another linear model. You can think of it like a classification analogue of linear regression. OK, so we’ll start by importing logistic regression from Saikat Learn. Next, we’ll create an instance of our model. Notice that I’m passing in one argument into the constructor called Si. Si controls what is called the regularization penalty. Basically, it’s a way to prevent the weights from becoming too large, which can be problematic. Again, this is a more advanced technique that you could learn about if you learned logistic regression in death. OK, so next, we need to convert our targets into binary format. Currently, they are just real numbers. We can do so by assigning the boolean array. Why greater than zero? We do this for both the train and test set to get C train and C test. So for our classification models we use X and C instead of X and Y. Luckily the API for Skillern models is pretty much the same no matter what model you use. So again, we call model Duffett on X train and C train and model that score on both the train set and test set. So let’s run this. OK, and we see that our model does slightly better than chance for both the train and test set. Next, we obtain our predictions by calling model to predict on both X train and next test to get P train and P test. Now, in case you’re curious, we can cast these to a set to check what the possible values of P train and P test actually are. So since we passed in booleans for the target, we get back booleans for the predictions. Next, we’re going to compute the return of our algorithm when we use this new model. Basically, this is just a copy of the code we had earlier where we assign P train and P test to the position column. Then we multiply the position column by the return column to get our algorithms return. Next, we’re going to compute the sum of the Al Gore return column for the Transat. OK, so we get about zero point five nine. Next, we’re going to check the some of the ALGOL return column for the test said. And we get about 20 percent. Next, let’s again prints out the buy and hold performance so that we can compare. All right, so we see that our classifier results in a slightly better performance than buy and hold.

8. Using a Random Forest Classifier for Machine Learning-Based Trading

In this lecture, we are going to continue our exploration of machine learning based trading. This lecture will consider the question, what if we plug in a super powerful classifier? If you’ve studied machine learning, you know that linear models are comparatively pretty weak at the same time. This doesn’t necessarily mean they aren’t useful. However, it’s worth trying a more powerful model just to see what happens. Recall my rule. All machine learning interfaces are the same. What this means is we can accomplish this with zero new code. All we need to do is plug in a different model. OK, so let’s start by importing the random forest classifier, as you know, the random forest is one of the best plug and play models for tabular data. This is because it combines the expressiveness of decision trees along with the variance reduction that we get from unscrambling inside the constructor we pass in random state equals zero. If you’ve studied random forests, then you know that there is some randomness inherent in the training process. Thus, if you don’t include a random state, you will get a different result each time. This does not imply that you should use the random forest this way. This is just so that we get consistent results in this lecture. In the real world, you would want your model to be robust enough to work all the time, no matter the random state. OK, so next we call the usual functions model Duffett and model that score. So we can see something very interesting on the train set, we get a perfect score, one hundred percent on the test set, we still only do slightly better than chance. So random forests are very powerful, but they are so powerful that they might fit to the noise in your data. In this case, that’s definitely happening. OK, so next, we’re going to come up predict to get P train and P test. Next, we assign the position column based on P train and P test, as usual. Next, we compute the ALGOL return column, same as before. Next, we check the some of the Al Gore return call on the train set. Not unsurprisingly, it does extremely well for point seven. This is because it’s perfectly predicting all the ups and downs on the train set so it’s able to take advantage of all the ups and never experiences any of the downs. Of course, the test set may be a different story. So next we check the some of the ALGOL, return them on the test set. And we get about thirty three percent, let’s print out the buy and hold results once again. So we see that the random forest greatly outperforms the buy and hold strategy for both training tests. Now, remember, these results are only for this particular random seed. This does not imply that you can take this algorithm and go make a billion dollars. Let’s scroll back up and try. Random state equals one. OK, so as you can see, these results are not robust to the random state, we still get a perfect score on the transfer, but now we underperform on the test set. Let’s see what happens if we try random state equals to. OK, so again, we get a perfect score on the train set, but now on the test set, our return is even higher at zero point three eight. So there are lots of important ways that you have to test the robustness of your models. As you just saw. Sometimes there’s randomness in the model training process itself. There’s also randomness in the data. The signals that we’re predictive 20 years ago or 10 years ago or even three months ago may not be predictive any longer. Remember that random forests are a very powerful model, but a model only works as well as the data you give it.

9. Algorithmic Trading Section Summary

In this lecture, we are going to conclude everything we learned in this section. Please note that our study of algorithmic trading is not yet complete. In the coming seconds, we will be studying reinforcement learning, which will extend what we’ve learned here. But this is a good stopping point to consider what we’ve learned so far since reinforcement learning is a very different subject. OK, so this section was all about algorithmic trading, we learned about a classic technique called trend following this made use of a pretty basic tool we learned about early on in the course called the simple moving average. Next, we learned about machine learning based techniques. We applied linear regression, logistic regression and random for us as our machine learning models. We saw that although random forests are powerful models in general, they also have the capacity to overfit to noise, which is abundant in financial data. I want to reiterate one of the important themes of this section and of this course in general, it’s that models are just models. You can’t say this model is good and it should work because it is a good model. You can’t say so-and-so says this is a good model. Therefore it will work for me. As we saw and have considered previously, your data also matters. We looked at the returns on a few components of the S&P as input features. Even considering only these, there is still many more choices. For example, you could use multiple lags of those returns. This will give you a vector auto regressive model. You can plug this kind of data into more powerful models like CNN Zain Arnon’s from Deep Learning. However, you can assume that CNN Zain Arnon’s will work better than Anan’s or linear models. And we see students make the mistake of presuming that deep learning is powerful. Therefore, deep learning should always work. The more powerful the deep learning model, the better. These are not good assumptions, especially in finance. Going back to data, there is still much more data from around the world that can be used, as you know, European markets and Asian markets open and close at different times compared to the U.S. market. Therefore, it’s possible that what happens in those markets might be predictive of what happens in the U.S. market. In addition, there’s always opportunity to make use of Twitter and the news using NLP techniques such as sentiment analysis. So that would be an example of having multiple machine learning models in your pipeline. Your first model reads the news and outputs the sentiment for that day. Your second model takes the sentiments and tries to predict tomorrow’s return. So these are just some ideas about data that you can consider. But also remember that if you do end up finding a good signal, other people can find it, too. And if everyone knows about it and everyone is using super fast computers to act on it quickly, then that is exactly an efficient market. Another important consideration is to check for robustness over time in our examples, we looked at only a single train set and test set. But what if I want a strategy that performs well all of the time this year, next year and the year after that? This doesn’t necessarily mean that you will use the same model with the same parameters over all time periods since you could just retrain your model periodically. It means that we care that the method itself is robust. That is, can I take the same strategy one year from now, train a new model and still perform well? Another important consideration is to check for robustness over different assets in our examples. We looked at only the S&P. If you want to consider our algorithm good, then you might want to make sure that it works for other signals as well. For example, we might see that it beats buy and hold for the S&P, but does it buy and hold for the Dow Jones? How much money does it lose if we use the algorithm on a stock that ends up going down? Can it handle a market crash? These are all important questions to ask. OK, so to conclude this lecture, let’s remember that these lectures are meant to introduce you to some ideas, but as with portfolio theory, it’s necessary to make sure your algorithm is robust and not just overly optimistic. In the next section, I will introduce you to a new paradigm called Reinforcement Learning. Reinforcement learning is a complex subject, and although I’ve included a multiple hour introduction to the topic in this course, it still might be too dense for you to take in and digest all at once. For many people, myself included, it’s necessary to study reinforcement learning in depth until you truly understand it. This includes coding everything from scratch and demonstrating all the ideas in a computer program instead of just thinking about them in your mind. Personally, that just does not work for me. Nonetheless, the next section will review reinforcement learning from scratch. After that, we will apply what we’ve learned to algorithmic trading.

6. VIP The Basics of Reinforcement Learning

1. Reinforcement Learning Section Introduction

In this section of the course, we are going to discuss the theory behind reinforcement learning, this lecture will give you an introduction to reinforcement learning and we will talk about it in general terms without any math or terminology. One thing you have to brace yourself for when you’re learning about reinforcement learning is how different it is to supervise then unsupervised learning. So if you just took an introduction into machine learning where you learned about models such as Naive Bas and K means clustering or you come from a statistics background, you will be surprised at how different reinforcement learning is compared to what you are used to. So try to take some time to soak up these concepts and don’t feel intimidated by this new and different way of thinking. I want to begin by thinking about supervised learning when we think about, say, an image classifier, you can think of it as a static function. I pass an image and I get a prediction. It tells me what kind of object is in the image. For example, there’s no notion of time. I pass in another image. I get another prediction. The image classifier is just a function. I give it an input and it produces an output. So what do I mean by static and what do I mean by time you might immediately think of recurrent neural networks, which are neural networks that can handle sequences, inputs that vary with time. However, this is not the kind of time I am thinking about. If I pass in some stock prices for a given time period and my model predicts whether the stock will go up or down tomorrow. That’s still a static function. Here’s what I mean by time. Imagine you are building a self-driving car simulation at each moment in time, your neural network can take a snapshot of the screen and decide what to do next. Should I steer left? Should I steer right? Accelerate or brake? So this is the difference between a supervised and reinforcement learning, supervised learning is just a function you can call this function repeatedly, but it’s still just a function you pass in an image and it produces an output reinforced. Maloni is more like a loop. It exists to achieve some goal, for example, driving you to your desired destination inside the loop. Yes, it still takes in an image and produces an output that specifies how to control the car. But importantly, this reinforcement learning program has the concept of time. It doesn’t think about just what is this image? How do I translate this image into an output prediction? Instead, it has the capacity to plan for the future. Even though at this moment the car may only see where it is on the road right now, it knows that there is some sequence of actions it must take in the future that will lead it towards its goal. All right. So that’s the major difference between a supervised learning and reinforcement learning with supervised learning. We have no concept of a goal or the future or planning. We just take an input and produce an output. It’s a static function with reinforcement learning. We have a plan and that plan can be carried out in the future to reach some predefined goal. Here’s another way to think about reinforcement learning. Think about the data with supervised learning again, let’s use image classification as our example. We must have a label for every input in our training set. So if we have an image of a dog, we must have another item specifying the class dog. If we have an image of a cat, we must have another item specifying the class cat. In other words, for every X we must have a Y. It’s important to remember that label data sets must be made by humans, sometimes students have this really funny idea that we should just automate the creation of label data sets. Guys, if we already had computers that could perfectly label data, then that would mean we have already solved machine learning. Computers that can perfectly label data are actually what we are trying to build. If such computers already existed, then we would not need to build them. All right. So hopefully you are convinced that label data comes from humans and not some super smart computer. Why is this important? Well, think about our self-driving car. Again, take this image of the road. If we were to use supervised learning for this data point, we would need to give it a target. But what’s the target? Should I steer left? Should I steer right? Should I accelerate? Should I break? In fact, you probably cannot give this image a target. And even if you could, how could you possibly label every single frame that the car will encounter along its journey? Imagine if you have a standard camera that captures images at 30 frames per second and you have a one hour drive as your data set. One hour is thirty six hundred seconds, which means you would have to label one hundred eight thousand images from just one trip. Instead, it reinforcement learning learns using goals rather than targets. For example, suppose you wanted to teach a reinforcement learning algorithm to solve a maze in this scenario. The goal would be finding the maze exit. You do not need to tell the algorithm what the correct thing to do is for each position on the maze, since that would be supervised learning. Instead, the only thing the reinforcement learning algorithm needs to know is what is the goal from there. Figuring out what to do in each position of the maze can be found by reinforcement learning. That is the power of this new paradigm.

10. Epsilon-Greedy

Earlier in this section, I told you about the value of treating the policy as a probability distribution. First, it allows us to describe the entire MDP as a set of two probabilities, one representing the environment dynamics and one representing the agent. This allows us to reason about the MDP mathematically and find a solution to both the prediction and control problems, as we just did. The other important reason, which I mentioned briefly earlier, is that in order to find the best action, we must first know the result of performing different actions. In other words, we must have collected those samples. Here is the problem when we take the Amax overall actions of, say, the que value here is not changing that much from one episode to the next, let’s say we have three actions. So we want to choose from que of as a one, QSA two and QSA a three, while perhaps we initialised que of as A1 and QSA two to zero while we initialised curious a three to one also assume that all the rewards are positive. During the process of playing the game, maybe QSA three gets updated to a value of two. We will now always choose Action three because that gives us the best value out of the three possible actions. The problem is we don’t know the true values of QSA one. And you have say two, because we never collected data from performing those actions. How do we know the values of the other actions if we have never actually performed those actions? In fact, we don’t know those values at all. We just happen to initialize them to zero. Because of that, there is no way for us to confidently choose the right action. This is a more general problem known as the explore exploit dilemma. Suppose you go to a casino and you’re playing a simplified version of slot machines. You have several slot machines to choose from and you don’t know which one you should play. However, you do know that not all these slot machines yield the same reward. As I said, these slot machines are simplified. So there are only two possible outcomes. You can either win or you lose. If you win, you get one dollar and if you lose, you get zero dollars. Obviously, you would like to play the slot machine in which your chances of winning are highest. OK, so how can you figure out which slot machine will give you the best chance of winning? Well, you cannot ask the manager of the casino because he wants to make money. He will not share that information with you. So what do you do? Your only method of figuring out which slot machine is best is to collect some data. So let’s say you play each slot machine one thousand times. The winner of the slot machine is simply equal to the number of times you one divided by the total number of times you played. But what’s the problem with what we just did? Let’s say you have to choose between five slot machines, if you played a slot machine 1000 times, that means you’ve played of 5000 games. Now, let me remind you that playing a slot machine at a casino is not free. In general, collecting data is not free. It usually requires time, resources or both. Let’s say each time you play a slot machine, you spend 25 cents. You need to win at least one out of every four times in order to break even. Imagine that for some of these slot machines, you win a zero times. Now it becomes clear that playing those slot machines one thousand times would be a huge waste of both time and money. Here is another question to consider why should we collect 1000 samples per slot machine? Why not 100? Why not 10? As you know, the more samples you collect, the better and more accurate your estimate becomes. Unfortunately, this also equates to more time and money spent. That’s why we call this a dilemma, the explore exploit dilemma means we are trying to balance both exploration and exploitation. We want to explore so that we can collect more data to find out which slot machine is truly the best. On the other hand, we want to exploit because we want to play what we believe is the best slot machine in order to win more often and to make more money. These two forces always oppose each other. The typical answer to this problem in reinforcement learning is called epsilon greedy. This allows us to be greedy and exploit what we think will yield the highest future, some of rewards. But we have a small probability, Epsilon, of choosing a random value so that we can explore and collect samples for our table to make our estimates of Q more accurate. And remember, they do need to be somewhat accurate in order for us to confidently choose the best action. Here’s how we will do this in code, instead of always performing the action specified by the policy, we will generate a random number between zero and one if this random number is less than Epsilon. We will choose an action at random. Otherwise we will perform the action specified by the greedy policy, which is greedy with respect to Q given the state s.

11. Q-Learning

At last, we are ready to study the famous school learning algorithm. Let’s recap what we’ve done so far because it’s been quite a long process to even get to learning. First, we defined all the relevant terms, such as Asian environment, state action, reward and so forth. Having these definitions allowed us to structure our problem mathematically. Specifically, we can model reinforcement learning problems as MDP Markov decision processes. Next, we considered how we would solve an MDP, meaning two things. First, we solve the problem of prediction, finding the value function a given a policy. Second, we solve the problem of control, finding the best policy in a given environment. We learn that if we know all the probabilities in an MDP, this is quite easy. But when we don’t know the probabilities, we can use sampling methods which we call Montecarlo. There is one major problem with Montecarlo, it is that in order to calculate returns, we must wait until an episode is over. This is because the return is defined as the sum of all future rewards. We can’t know the sum of future awards until we have collected them, and therefore we must reach a terminal state before calculating the sample returns. Why is this a problem? Well, imagine the scenario where you have very long episodes or the scenario where there is no terminal state. In the latter case, Monte Carlo is not an option in the former, it’s still not ideal because that means your agent has to spend a very long time performing suboptimally, even though it has already collected a lot of data from which it may improve. The answer to this is temporal difference methods, if you recall I said earlier that one of the most important features of the return is that it can be defined to recursively the return at time. T can be expressed in terms of the return of time T plus one. You saw how this helped us both create and solve the Belmond equation. Now it’s going to help us once again. One simple way to think about it is this. Monte Carlo methods were an approximation to an expected value problem, several different methods are simply an approximation to Monte Carlo. So in other words, they are an approximation of an approximation. Recall this one neat trick with Montecarlo methods when we’re updating V, which is the sample mean of all the samples we’ve collected, we don’t have to express the update as a sample mean. That would mean we have to keep all the samples around, which can take up lots of memory and take lots of time to calculate, instead we can express the current sample mean in terms of the past, the sample mean, we noted that this looks a lot like gradient descent. Well, why not test out this theory, let’s set our squared brigade to be the squared error between my true target G and my value for the state s VLS. Now, let’s say I want to update viewers using the latest sample G, which I’ve just collected in order to perform the update. I’m going to use gradient descent set Vivas to be the old value of your voice, minus the learning rate times the gradient. Well, what is the gradient? If we plug this gradient into our gradient descent update and ignore the two, since it can be absorbed into the learning rate, we get back the exact opposite equation we would use for the exponentially decaying average. As a side note, I want to mention that there is ultimately no difference whether we call what we are doing gradient ascent or gradient descent, because there’s a plus sign here. You might think of it as gradient ascent. The plus sign is natural to use if you derive this expression from the sample mean update. But if you derive this expression from the squared error, you will get a negative sign and you might think of that as gradient descent. Ultimately, however, this is just basic algebra and you should confirm to yourself that both expressions are equivalent. So what’s so significant about this? Well, now we’re going to put together these two ideas. Idea number one is that updating the value function, using the exponentially the king average is just like gradient descent. Idea number two is that the return? It can be defined recursively. So how about this, we’re going to continue using gradient descent, but instead of using the full return, we are going to estimate the return. We collect the next reward are, but instead of waiting to collect any future rewards, we simply guess that they will be close to Vivas prime the value of the state where we landed. So instead of using Jey equal to our plus GAMMER times, the next reward, plus Gammer squared times, the next reward and so on, we recognize that guy is just equal to our plus gammer times the next return G prime. But G Prime has the expected value via prime. So we instead just say, gee is approximately equal to our plus gammer times as prime in this way we only have to wait one step before updating our model. We no longer have to wait until the end of the episode. We call our plus Guerma Times Vivas prime, our bootstrap estimate of the return it allows us to update via immediately after obtaining the next reward are instead of having to wait until we’ve collected all the future rewards. This method is called the temporal difference method. Here’s some pseudocode so you can conceptualize how this will work as a side note, this should also give you some insight into what goes on inside our play episode function, which we haven’t really discussed yet. And this pseudocode, we don’t have any need to abstract away the play episode function because playing the episode is part of this loop. We have things to do on each step of the episode, and therefore we cannot encapsulate it or delegate it to some other function. To start assuming we are given some environment and some policy, we initialize the value function to be random. Then we enter a loop, which plays a predetermined number of episodes. Alternatively, you could also run the loop until you find that Vivas, Convergys or in other words, settles on some value and doesn’t deviate from it. Inside the loop, we begin playing an episode, the first thing we do is call Ian Varicella, which resets the environment and puts us back into the initial stage and returns that initial state. We’ll call it S. Next, we initialize a done boolean flag to false, this boolean flag will get set to true when we’ve completed an episode. Next, we enter a while loop that completes when Dunn becomes true inside the loop, we grab the action from our policy. We then perform the action in the environment by calling Ian Vedat Step, this returns three things the next state as prime, the reward are and the next one flag note that I’ve designed this pseudocode to be similar to the opening I Gympie, which has become somewhat standard over the past few years. It’s easy to understand and it will help you in the future if you ever do start using open air. Jim, the reverse is also true. If you’ve had experience with open a Jim in the past, that should make things easier to understand. Next, we do the big update, the temporal difference update we’ve been discussing throughout this lecture, finally, and this is important not to forget, we must update the variable s for the next iteration of the loop, what is currently the next date as prime will become of the current state? S in the next iteration. Lastly, when the loop is complete, Vivace has converged. There is one odd thing about temporal difference learning. Look carefully at this so-called gradient descent update. The target is R plus gamma times Vivace prime. The prediction is Vivace. If we relate this back to supervised learning, we noticed something strange in supervised learning. We are given the target as part of the data set. But here we are doing something surprising. We are predicting the target itself. Part of the target is given, that’s the reward are, but the other part of crime is actually a model prediction. Thus it’s more correct to say that what we are doing is not quite ready in the sense it’s called a semi gradient instead. But this is just a name. It is the principle that’s important, the principle is we don’t know the true target. We are estimating it. And this makes it very different from supervised learning. All right, so what we looked at so far was the prediction problem. Now it’s finally time for the big reveal. We are going to solve the control problem using the famous Q learning algorithm. At this point, we spent so long building up the prerequisites for learning that you should hardly be surprised at what you see. Nonetheless, let’s have a look. As before, because Q Learning is a control algorithm, we are interested in updating. Q Rather than updating V at a high level. We are mostly interested in the innermost part of the loop. That is where we choose an action and take a step in the environment and where we update the cue table. So these are the two pieces we are going to focus on here when we choose an action. We are again going to use an epsilon greedy approach. So with small probability epsilon, we will choose a random action. Otherwise we’ll take the arguments over. Q Given a state s. Once we choose our action, we then take that action in the environment. When we update you, we do something very subtle but important, that is when we calculate the target, we ignore whatever action we are going to take next. Instead, we assume that we will take the greedy action and take the max over. Q Given the state as prime, this has two advantages. First, it means that in order to update. Q We don’t have to wait until we obtain the next action a prime in the next iteration of the loop. Second, it makes Q learning what is called an off policy algorithm. This means that I can freely explore by my algorithm, will update the cuttable as if I had acted greedily. So here’s what Kyul learning looks like when we put it all together. Luckily, it looks quite similar to the prediction problem pseudocode. First, we are given some environment objects. Then we initialize CU with random values. Then we enter a loop that goes for a predetermined number of episodes. Optionally you can keep going until Q or the policy converges. Next, it’s the same as what we had before we reset the environment and start back at the initial state, we initialize the Dunn flag to false then and we enter a loop for the episode, quitting only when we are done. Next, we use Epsilon greedy to choose an action. Then we take a step in the environment. We get back the state as prime, the reward are and the done flag. Next, we create the target for the Q update taking the max over. Q Given the state as prime. Next we update Q of Santé using the target. We just calculate it. Lastly, we update the current state? S to be the next state as prime. Finally, when we exit the loop, we have found the optimal policy. Let’s summarize what we just did since that was pretty long. First, we started by noting that Monte Carlo will not work for infinitely long episodes and that it’s problematic in that we have to wait until an episode is over in order to do any learning. Instead, we make use of the fact that the return can be defined recursively. This allows us to approximate the return using only the next reward and the value at the next state. We also learn that reinforcement learning ultimately starts to look like supervised learning, where our target is not a real target, but rather an estimation made by our own model. We are essentially doing gradient descent on a cuttable using this approach. It allows us to update our cuttable on every step because we only have to wait until we receive a single reward to make an update. We call such an approach online learning because the agent learns while it collects data.

12. How to Learn Reinforcement Learning

To end then summarize this section, I want to discuss at a high level what it takes to learn at reinforcement learning for real, personally, I think it would be quite difficult to grasp reinforcement learning and understand it on an abstract level from just a single section of a single course. Let’s start by recognizing that reinforcement learning is very different from supervised and unsupervised learning. Nowadays, when you’re studying supervised then unsupervised learning, many beginner courses take the approach of avoiding how to implement the models. This is not a good approach because it doesn’t prepare you for understanding real machine learning. Let’s suppose the structure of the courses like this. First, you discuss the intuition behind how the model works, just some pictures and some descriptive ideas. Then you learn how to use that model inside. You learn not including loading in the data or looking at the results. So what is the problem with this approach? Well, it allows you to not only make many mistakes and have serious misunderstandings about the content, but perhaps the biggest problem with this approach is that you will never know what those misunderstandings are. In other words, you are in a place where you don’t know what you don’t know. You don’t know your mistakes, and hence you can’t fix them. You’re confident that you’ve learned something. But this is nothing but blissful ignorance. And again, these are just my observations of other students. So don’t take it as a personal judgment of you. This is not so much a problem, if that’s how far you want to go, if you only ever plan on using APIs and you never want to become an actual professional, that’s totally fine. And nobody is judging you. But it becomes a problem when it’s time to stop using APIs. This turning point appears when you want to learn how to do reinforcement learning. There is no API for reinforcement learning, at least not yet. So what is one to do? Well, you have to implement those algorithms yourself. The problem is, if you have no experience implementing machine learning algorithms, reinforcement learning is not a good place to start. I want to also comment about the many blogs, tutorials and so forth that exist out there attempting to teach you about reinforcement learning, consider how long it took us in this section to go from nothing to learning at a pace that I would consider as fast as possible. Compare that to the length of a typical blog tutorial. It should be clear that it would be impossible for a short blog post to include enough detail to get you to the point where you can actually implement reinforcement learning yourself. The best advice I can give you if you want to learn reinforcement learning for real is this I would recommend taking a full course or multiple courses about reinforcement learning, learn about basic tabular reinforcement learning before you move on to approximation methods that will make understanding the concepts much easier. In this section, we spent most of our time discussing the concepts under the assumption that we were using a cuttable and not a neural network. You want to learn about the basic approaches to reinforcement learning, which can be categorized into number one dynamic programming methods, number two, Montecarlo methods and number three, temporal difference methods and make sure to spend some time on implementation. After that, move on to approximation methods with linear models, once you’ve done that and you’ve implemented a reinforcement, learning algorithms with linear models move on to using deep learning for function approximation. At this point, you should be very comfortable with the concepts, but implementation is still another matter entirely unlike supervised in unsupervised learning, reinforcement learning is very hard to get right. Even if you are an expert programmer, it is very easy to introduce subtle bugs that are difficult or nearly impossible to track down. And of course, you don’t have to take my word for it. Here is a post by Andre Carpati, probably one of the most famous reinforcement learning researchers talking about just how difficult it can be in this comment. Another user talks about how they spent an entire year trying to get Q learning to work with neural networks for a particular game. Many students who take my courses get frustrated when an exercise takes more than a few minutes. Now, imagine working on something for one year. Andre response to this comment and says it took me six weeks to get policy gradients working, and keep in mind, he’s one of the world’s leading researchers in reinforcement learning. He has all the state of the art technology he needs on demand, and he’s surrounded by other researchers who are just as smart as he is so he can ask them for advice or to check his code at any time. So the key idea I want you to keep in mind is that as you learn reinforcement learning realized that implementation is very non-trivial even for experts. You should expect to spend a lot of time and effort trying to get things right.

2. Elements of a Reinforcement Learning Problem

In this lecture, we are going to discuss reinforcement learning from a more technical standpoint, and this will allow us to define most of the terminology involved in reinforcement learning problems. I’m a big believer in learning by example. So in this lecture, it’s not going to be so much about abstract and technical definitions as it is about providing examples of everything. Let’s start with the main objects in a reinforcement learning problem, the Asian and the environment. The best example of this is yourself. You are in Asia and the world is your environment. Maybe your long term goal is to ace your math exam. And so just like an autonomous vehicle driving to a destination, you must observe your environment and make the correct decisions every day in order to achieve your goal. That means studying, going to class, taking notes, doing your homework, asking questions when you are confused and so on. So that’s one basic example. Here’s another example that’s closer to what we might actually use reinforcement learning for. Suppose you’re writing a program that plays tic tac toe. What’s the Asian and what’s the environment in this case? The environment is composed of the computer program that implements this tic tac toe game. Of course, this computer program may also involve some form of A.I. that will be the other player in the game, but for all intents and purposes, just pretend it’s a bunch of if statements and predefined rules. So it’s not intelligence per say, it’s just a computer program written by someone else, just part of the greater tic tac toe program. You can imagine that this tic tac toe program will have an API that will allow you to interact with it programmatically, so, for example, there might be a function to start a new game. There might be a function to place your X or your O at some location on the board. There might be a function to read the state of the board. So you can see where all the X’s and O’s have been placed so far. There might be a function to check whether the game is over and if so, who won the game, the computer or your agent. So that’s the environment. Your agent, on the other hand, will be another computer program that interfaces with the tic tac toe game. Your agent may use an algorithm such as one from reinforcement learning in order to learn how to play tic tac toe. From experience, it makes use of the API to interface with the environment. So here we have a program that plays the game. Part of this code is where the agent reads the state of the game board and chooses the most intelligent action. That’s our A.I. represented by our agent. Here’s another popular example, video games, this is a famous classic Atari game known as Breakout. By the way, if you don’t know how this game works, there are many places where you can play this game online for free. So if you’ve never played this game before, please go and give it a try. In this game, the environment is obviously the game itself. The goal is to clear all the blocks and your job is to move the paddle in such a way that the ball destroys the blocks but never falls to the ground. The agent would be your computer program, which can read information from the game like what the screen currently looks like so it can figure out where the blocks are with a paddle is where the ball is going and so forth. Its job is to control where the paddle goes. So basically you can move left, right or do nothing. Next, let’s continue defining more terms so far, you know, about the Asian and the environment, the next term I want to define is episode what happens when I play a game of tic tac toe or breakout? Well, some sequence of events will occur and then at the end I will win or lose. With my math exam example, you will take your math exam and then you’ll get your grade. Now, we know that with learning algorithms, the way that they learn is with data. So if you’re training a dog versus cat classifier, you’ll need lots of labeled images of dogs and cats. Similarly with tic tac toe or breakout, once the game is over, I can opt to play again. This is the method through which I will gain experience or to be more technical data. You might call these games or rounds or matches, but in reinforcement learning, the official term is episode. So when you’re training an agent to play tic tac toe, you’re going to play multiple episodes and at the end of each episode, your agent will have won or lost, hopefully by the end of the training process. Or in other words, after many episodes, your agent will be winning more than losing. Of course, not all reinforcement learning environments are episodic, to say an environment is episodic means that they end at some point and you can start again with the new fresh episode. Furthermore, there is no relationship between one episode and the next. So the fact that I lost the previous tic tac toe episode will have no effect on the environment in the next episode. However, there are examples of non episodic environments. Take, for example, the stock market, for all intents and purposes, this can go on forever. There’s no real notion of the end. Well, if you lose all your money, then technically there is nothing more you can do. But it’s not the same as losing a game of tic tac toe and starting again. You can’t go back in time and restart the stock market. Another example is an online advertising system. Your agent’s job will be to choose the right advertisements, to show to users at any given moment in order to maximize your company’s revenue, it should do this continuously. There’s no concept of an end to an online advertising service. All right, so those are some examples of some non episodic environments, we can refer to such environments as having infinite horizons. All right, so to recap the terms we’ve discussed so far, now we have Asian environment and episode, the next few terms I would like to think about are the state action and reward these items. Help us describe what goes on when the Asian and environment interact. Once again, use our tic tac toe example in this scenario, the state would be the configuration of the board. So for each position on the board, I want to know, is there an X there? Is there an O there or is it empty? This information is all that I need in order for my agent to make an intelligent decision about what moves to play next. Speaking of which, the moves that the agent makes are what we refer to as the action, so in tic tac toe, to take an action would mean placing a new X or A. somewhere on the board. Finally, the reward is just a number that you can receive at any moment as you play an episode of the game. By the way, keep in mind, when I say the word game, I don’t necessarily mean a board game like tic tac toe or a video game like breakout. When I say game, I mean, it is more of a generic sense. In any case, perhaps the reward you get in tic tac toe, maybe plus one for winning, minus one for losing and zero for draw. Although this is just an example in general, you can always assign rewards yourself in order to improve the training of your reinforcement learning agent. Here’s another example of states actions and rewards. Think of a maze in this maze. The state is your position in the maze. Your actions may consist of the various directions. You can go, for example, up, down, left or right. The reward is tricky. Remember, I said it’s up to you to think of a good reward to assign to your agent to encourage it to learn how to solve the environment. You might say plus one for solving the maze and zero otherwise, but ask yourself, is this a good strategy? Imagine you throw your agent into this maze and it has to learn what to do. Imagine your agent has played this game 10000 times and has never solved the maze. We can pretend the environment is episodic so that after taking one hundred steps, you reach a terminal state and the game is over. What happens if we get zero reward each time? Well, in that case, the agent learns that it does not matter at all what it does because doing anything always leads to the same reward zero. In this case, the agent has no incentive to solve them is your agent will never prioritize one action over another because it knows that no matter what it does, it always gets zero reward. In this case, all actions are equal. Perhaps a better reward structure would be to assign minus one reward at every state. In this case, you can maximize your reward by solving the maze as fast as possible, doing any extraneous actions will lead to a more negative reward. Not solving the maze will lead to the most negative reward. So in this case, assigning a negative reward upon reaching any state will allow your agent to solve the environment. Now, you have to keep in mind this is not English class, so you have to remove any bias you may have about what connotations are associated with the term reward. You might think of reward as a good thing, like a prize. For example, if you’re a dog and you’ve just successfully completed a trick, your owner may give you a treat as a reward. But in reinforcement learning, this is not what we mean by reward. The only constraint is that the reward is a real number. It can be positive, negative or zero. You will also receive this number at every step in the environment, not just when you reach some goal or definitively fail to achieve it. The agent, as you will learn later in the section, will try to maximize its reward over each episode. For example, you may get a reward of minus one hundred. This is better than a reward of minus one million, maybe minus one hundred. Reward corresponds to successfully solving the environment, but in the end it’s just a number. Don’t associate minus 100 with negative connotation and plus one hundred with positive connotation. So just remember this, the reward is not a prize, the reward is a number which is to be maximized. You can think of it as kind of the opposite of a lost function, whereas we want to minimize the loss in a supervised or unsupervised learning problem and reinforcement learning. We want to maximize reward. Since I love examples, here’s one more. Imagine again the game breaking out in this case, we actually have several options for the state. For example, we may have perfect information about the game. We could be told the exact positions of all the blocks. We could be told the position and velocity of the ball. We could be told the location of our paddle, and we could be told our current score and the number of lives we have left. Although I think you’ll find that most reinforcement learning applications do not make use of such information. Another way you can read the information about the state of the environment and breakout is to look at the games RAM, in other words, look at the values it has stored in memory in contrast to the above. This actually is one method used in modern reinforced military applications. It’s a proxy to the above perfectly defined state. You can imagine that it should be quite possible to derive the locations of the blocks and the position and velocity of the ball and so forth from the values stored in RAM. Although I think the most common way to represent the state in contemporary reinforcement learning is to use screenshots from the game and this way our reinforcement learning. Ajam is learning to interpret images of the video game just as we do as humans. I think this is the most meaningful because it’s the closest match to how you and I play video games. We look at the screen, you can imagine that models like Convolutional News that works would be useful here. One complication that can arise from only looking at images of the screen is that you don’t really have information about movement and image is only a frozen picture of the game at a single point in time. Looking at this image, how can I tell which direction the ball is moving? And so this allows us to consider an important point. The state need not be what I observe in the environment. It can also be information derived from both current and past observations. So one way of dealing with the problem of frozen pictures is to simply include past frames as well. In the famous discussion paper, they use the most recent four consecutive frames of the game to represent a single state. To get back to our states actions and rewards, the rest of it is quite basic. The actions consist of the various moves you can do in the game. You can think of this in terms of pressing buttons on a joystick or a control pad. In breakout, you can move the paddle left or right. For the reward as an example, you might get plus one reward every time you destroy a block. As a final note in this lecture, I want to introduce you to the concept of state spaces and action spaces. This is important as we move down from high level ideas and concepts to the actual math that will allow us to solve reinforcement learning problems. The particular math concepts that we need to describe state spaces and action spaces is the set. The stage space is the set of all possible states and the action space is the set of all possible actions. We don’t need to go further than this. We just need to know what it means. So as an example, consider the canonical example of a reinforcement learning problem known as grid world in grid world. The idea is you’re going to start in the bottom left square and your goal is to arrive at the top right square with a ruby is if you make it there, you get a reward of plus one. Below that, there is a losing state where if you arrive there, you’ll get a reward of minus one. And in the second row, 2nd column, there is a wall, meaning that your agent cannot go to that square. So that’s the basics of the game to describe the state space. That’s simply the set of all possible positions on the board. So you may want to pause this video and look closely at this list of coordinates to confirm that they correspond to positions on the board. The action space consists of the actions up, down, left and right. Now, the reason we had to talk about Griddled for a bit is because for our other examples, such as tic tac toe and breakout, the state base is much more complicated. The auction spaces are pretty simple since for tic tac toe. It consists of all the possible positions you can draw an extra reneau and for breakout it consists of moving left, right or doing nothing. But for tic tac toe, the state’s base is quite large as there are many possible configurations of the board as an exercise, I would strongly recommend trying to write a computer program that can enumerate all the possible configurations of a tic tac toe board. This should give you some intuition about why games like Chess and Go are very difficult. You can imagine that if a three by three board with only two possible characters can have thousands of states, imagine how many states are involved in chess and go. For breakout, the number of states is even larger. It’s equal to the screen resolution multiplied by the number of possible colors per pixel two to the power 24 or two to the power, eight to the par three. But for all intents and purposes, we can consider images just like Time series to be continuous valued. The only reason they appear to be the screen is because computers have finite precision and so the values need to be quantized when we have continuous values. This means that the number of possible values is actually infinite and in fact it’s possible for actions to be continuous also so that the action space is also infinite. Since this lecture was quite long, let’s summarize everything we learned, this lecture was all about defining some reinforcement learning terminology to help us in our discussion of reinforcement learning. First, we define the terms, Asian and environment, you can think of the environment as the world or whatever computer game you are teaching your agent to, when you can think of your AJAM as your computer program, the one that does the learning. Next, we define the term episode. This is like one round or one match of a game. As you know, machine learning models learn through data or in reinforcement learning parlance experience. And so you can imagine that in order to sufficiently learn how to play a game, this is going to require multiple episodes. Next, we learn about states actions and rewards, rewards are a number which can be any number, positive or negative. The job of a reinforcement learning agent is to maximize its reward. Actions are what an agent does in an environment, for example, playing a moving tic tac toe are going left or right in a video game. States are what we observe from the environment, but they can also be values derived from those observations or even a sequence of past observations. To add a little more to this, we call the last state of an episode a terminal state. So when you reach a terminal state, that is the end of your episode. Lastly, we define the terms state space and action space. These are the set of all states and the set of all actions, respectively. Using these terms, we can now talk about reinforcement, learning incoherently and build a framework that allows us to solve reinforcement learning problems.

3. States, Actions, Rewards, Policies

In this lecture, we are going to discuss the concept of states actions and rewards more in-depth. This lectures about how we encode states and actions when we are programming. Talking about states actions and rewards theoretically is nice, but eventually we’re going to have to put this into code. As a side note, since rewards are just numbers, this is trivial. There is no need to discuss how we represent numbers in code. Let’s start with the state, as mentioned previously, the state can be discrete or continuous in a game like tic tac toe. The states are discrete because they are just different configurations of the tic tac toe board. If we make a robot that has sensors like a camera, a microphone, a gyroscope, proximity sensor and so forth, those would all be continuous values. So you would end up with a vector of continuous values. In fact, this takes us back to regular supervised learning, if our targets are categorical, how do we represent them in code? If there are categories, we will use the integers zero up to K minus one. So we might see that dog equals zero, cat equals one, equals two and so forth. Obviously, it does not matter which category is assigned which number. And the reason we want to do this is because at some point we’re going to have to use these categories as array indices. And so similarly, if we have big states, then we’ll represent them in code using the integers zero up to a big S minus one. For continuous values, it makes sense to store them in a vector, although if you have something like an image, then it will be a three dimensional sensor. So more generically, if you have continuous values, you can think of them as a Tenzer with one or more dimensions. Now that you know how to represent states and actions in code, it’s time to talk about policies. When I first started learning about reinforcement learning, I found policies to be kind of an odd concept. Well, actually, I found all of reinforcement learning to be kind of odd and foreign, but you’ll get used to it. The idea of a policy makes sense at a high level, but it becomes somewhat ambiguous when you start thinking about how to represent it mathematically or in code. The policy is what the nation uses to determine what action to perform. Given a state, it’s important to keep in mind that the policy yields in action using only the current state. It doesn’t use any combination of the current state and previous states, and it doesn’t use any information about rewards. Technically, as I mentioned earlier, the state may be made up of multiple observations and that may also include the reward, although that is unconventional. But strictly speaking, the policy will yield an action using only the current state. The simplest way to think about a policy is that it’s a dictionary mapping or a function that returns in action, given a state here is such a function, as you can see, the only input is a state s and it returns an action a where the state s was the key to the dictionary and the action A was the value. The real question is how can we represent this mathematically? This is why it’s useful to talk about how we encode states and actions first, so imagine again we are in grid world. Your agent now has a dictionary representing what action to perform given the state, as you can see here. So, for example, if we’re to the left of the goal state, then the appropriate action is to move. Right. And just to be clear, the left of the goal state is zero to. If we’re at the initial stage, then the appropriate action is to move up and just to be clear, that is the state to zero. Now, from the initial state, moving right is just as valid since we can still reach the goal from that point. You can see that although I’ve encoded the states here explicitly as tuples, we could gain more efficiency by encoding them as integers corresponding to the tuples and using those integers to index an array. As you may recall from your computer science studies, indexing arrays is faster than indexing dictionaries. Thinking about policies as dictionary mapping is somewhat limited. There are two reasons for this. First is that this won’t work. If you have an infinite state space, you would need an infinite sized dictionary, which is not possible to have. Second is that it doesn’t allow our agent to explore its environment. Think of training your agent like teaching a baby. At first a baby knows nothing. It must try new things in order to figure out how the world works and build up its intuition. A reinforcement learning agent is the same way. If it has a fixed policy and it only does the same thing all the time, then it can’t gain new experiences. Thus, it makes sense for policies to be stochastic. Stochastic is just a fancy word for random. In other words, a more general way to represent policies is to represent them as probabilities. Representing policies as probabilities actually solves both of the problems I posed above. Let’s see how. First, it deals with this problem of randomness, the common way of dealing with this in reinforcement learning to allow the agent to explore is to give it a small chance of performing a random action. So here is a python function that can accomplish this. We first generate a random number. If this number is less than some small number Epsilon, let’s say zero point one, then it will choose an action at random from the action space. Otherwise we will grab an action from our fixed policy dictionary mapping. This method is called Epsilon Greedy, and you’ll learn later in this section why it’s useful and what the relevance of exploration is. So what about continuous state spaces, in fact, seeing policies as probabilistic easily lends itself to continuous or infinite state spaces. Imagine your state is a vector s. Now imagine we have some policy parameters. W The shape of W is the dimensionality of the state space by the size of the action. Space for now will presume that the action space is still categorical. So what do we do when we want to output probabilities for a set of categories? And in fact this is just like classification. We can use the softmax function. So now our policy is the softmax of W dotted with the state. S As you can see, this allows us to introduce a little more notation. It’s common in reinforcement learning to denote the policy with the symbol PI, not to be confused with the number PI for any given state. We can calculate a probability distribution over the actions base pay of a given s then to decide which action to perform in the environment, we can simply sample from this distribution. This allows us to explore if necessary, but we can still treat this policy deterministically if we want it simply by using the Amax. Also note that it’s not necessary to use a linear model, as we are doing here, we can, in fact use any function approximate here, such as a neural network. Now, at this point, you may be wondering, how can an intelligent agent possibly know what to do using only the current state? This is the problem we described before, just by looking at a still image of the road. How can I know what the correct action is? If you’re more inclined to think inside a supervised learning paradigm, you may insist that there should be a target here so that the agent learns to associate this date with this action. But in fact, it’s quite possible for an agent to learn how to plan for the future using experience collected by playing multiple episodes. Even without an explicit target for a given state, the agent can still learn what action to perform such that it maximizes rewards in the future. This is what we will learn about in the coming lecture’s.

4. Markov Decision Processes (MDPs)

Previously, we learned about terminology, the various words we use to describe reinforcement learning problems which allow us to solve reinforcement learning problems, now that you understand concepts such as agents, environments, policies, states, actions and rewards, we can build on this. The goal is to have a framework which we can then use to find solutions. So at this stage, we are still working to more accurately and more narrowly define a problem. Once we have an accurately defined problem, we can work within this framework and whatever assumptions it involves to derive solutions. The main assumption we make in the reinforcement learning is the mark of assumption, this is something we often discuss in terms of Markov models and sequence modeling. But let’s review it here anyway. The Markov assumption goes like this. Suppose we want to predict whether tomorrow will be rainy, sunny or cloudy. Perhaps your idea may be to base this on whether it was rainy, sunny or cloudy in the past seven days. Well, the Markova assumption is that tomorrow’s weather, it doesn’t depend on all of the past seven days, only the immediate previous day. Here’s another example of the Markov assumption, suppose I want to predict the next word of a sentence, I tell you that the previous word in the sentence is lazy. The mark of assumption is that the next word only depends on the previous word. Therefore, if the mark of assumption is true, then you should be able to predict the next word in my sentence. Of course, things are not so easy, you might think, because you are taking a course by the lazy programmer that the next word in the sentence is programmer. But in fact, that’s not what I had in mind. Now, let’s suppose I tell you the full sentence so far is the quick brown fox jumps over the lazy. Of course, we know that since we’ve seen this example many times that the next word is dog. So you might think this Markova assumption thing doesn’t really seem to be a great idea. In fact, there has been some work in reinforcement learning where they do not make use of the mark of assumption, although that is outside the scope of this course. The mark of assumption has actually been quite successful so far. In general, the Markov assumption states that the probability of the state at time t depends only on the state at time T minus one and not on any state that came before that. Now, by itself, the Markov assumption is weak. But as you recall I said earlier that we can make of a state whatever we want. So if we want to make the state three or four words long, that’s fine, too. The words are merely observations, but the state is made up of a sequence of observations. In this way, the Markov assumption is not as bad as you might initially think. So why do we need to know about the mark of assumption this is because reinforcement learning problems are commonly described as a Markov decision process or MDP. Previously, we discussed the Markova assumption in terms of a state only. But as you know, reinforcement learning problems involve other objects as well, namely actions and rewards. So the way we describe in MDP is to use the state transition probability. It’s the probability of arriving in the state at time T plus one and getting the reward attempt plus one, given the state at time T and taking the action at time t, another simpler way of writing this without time indices is just to write Piaf’s prime and are given as a note that because the reward are has no prime symbol, the prime symbol does not indicate time T plus one. You get the reward at times plus one for arriving in state as prime. But we do not put a prime symbol on our. So I just showed you the most general way of writing down the state transition probability, but often we can make it less general. For example, if we are solving a maze, then most likely we are going to make the reward deterministic. In other words, there’s no need to represent it as a probability distribution. In this case, we can use the notation PVS prime given DNA and the reward. It can be a symbol all by itself, usually denoted as are of S.A.S. prime. This encodes the idea that we were in status, we the action and we arrived in the next state as prime. We can even just say Ahvaz or Ahvaz prime in the case where the reward depends only on the state where you arrive at, which is actually quite common. An important point to consider is what is the usefulness of the state transition probability? You can imagine that if we are playing some game like breakout on Atari, it is very unlikely we will ever be able to calculate these probabilities, given that the state space would be infeasible to enumerate. And in fact, for Q Learning the main algorithm we’re going to discuss in the section, this probability is not used at all. I want you to think of the MDP and the state transition probability as stepping stones, they are simply conceptual tools, which we will use to further advance our knowledge and take us to a point where we can actually come up with a practical algorithm for reinforcement learning. In other words, while we’re not going to be using state transition probabilities directly in learning, they do help us build on what we’ve done so far so that we can actually arrive at coloring in a logical manner. Why else is the state transition probably useful? Imagine a game like tic tac toe, you might think there is nothing probabilistic about this game when I write down an X or no, that’s where the X or the O goes. Why is there a probability associated with that? Why is my action not deterministic? And in fact, it is entirely possible for your action to deterministically bring you to the next day. Imagine, for example, a classic test known as the inverted pendulum and this reinforcement learning task, your job is to control an upside down pendulum so that it does not fall down by moving the car left or right as necessary. Now, you might think to yourself, how do we describe such a system? Well, we use the laws of physics and now think to yourself, are the laws of physics not deterministic? For example, when we learn Newton’s three laws of motion to those laws of motion involve probability, the answer is no. So then what in the world do we need probability for? The answer is that your state may not completely capture all the possible information about the environment, consider tic tac toe again in tic tac toe. There is another player that players moves cannot be predicted by a tic tac toe Asian. Therefore, there are multiple possible moves that could occur between the agents previous move and the agent’s next move. If we’re talking about physical systems, we also have to take into account chaos theory. That is, even if you know the exact laws of motion. This does not mean you can accurately predict the future. In fact, the further into the future you try to predict, the more unreliable your predictions become. Sometimes we refer to the transition probability as the environment dynamics, which makes sense when you think about it in the context of physical systems. A system like an inverted pendulum is in fact a dynamical system. The last thing I want to mention in this lecture is to bring us back to this picture, which you’ve probably seen several times at this point, and MDP or a reinforcement learning problem consists of these two objects, the agent and the environment going back and forth. The agent reads the state from the environment and decides what action to take. It takes that action in the environment. The environment is updated based on that action and brings the agent to the next step while also returning an associated reward. The agent can then read this next day, take the next action and so forth. So they just go back and forth. And the circular pattern. What we’ve done so far is to represent both of these objects with probabilities. The environment is represented by the state transition probability PVS primates are given us and the agent is represented by the probability of a given us. This is more helpful than you probably realize at this point by representing both the agent and the environment as probabilities and allows us to describe reinforcement learning problems mathematically in particular, once we have an equation, we can solve that equation. Without an equation, there isn’t really anything to solve. That’s a pretty deep inside. In order to come up with a solution, we have to have a well-defined problem using mathematics specifically, probability allows us to create this well-defined problem. And that’s the first step towards finding a solution.

5. The Return

In this lecture, we are going to define a one more term which deserves its own lecture so far. I told you that the goal of the agent is to maximize the reward it gets. But as you recall, the reward may be structured differently in different games. For example, in tic tac toe, you may receive a plus one for winning or a minus one for losing. On the other hand, if you’re solving a maze, you might get a minus one at every step. So what does it mean to maximize the reward? Does it mean to maximize the reward at the next step or does it mean to maximize the total reward over an entire episode? Here’s the answer to stated more accurately, the goal of the agent is to maximize the sum of future rewards. Why is that? Well, it can’t maximize the rewards that got already. Those are in the past. They cannot be changed. Furthermore, we don’t want to only maximize the reward on the next step. What if we are solving a maze and we get a minus one for every step we take? In that case, the agent isn’t incentivized to do anything useful because no matter what it does, the immediate reward is still minus one. Thus, the agent’s true goal is to maximize the sum of future rewards until the episode is over. In this way, the agent is planning its future steps as well. It must have some concept of where it will end up because that’s the only way it will know how to maximize rewards in the future. To take a real world example, consider again the idea of preparing for a math exam for a math exam, you don’t receive any reward until you’ve completed the exam. The reward signal is your grade on the exam. But imagine all the actions it will take to actually maximize that reward. You’ll have to study. You have to do homework. You have to forego socializing with your friends. In fact, all of those actions do not sound very rewarding at all. And thus, if your only incentive is immediate gratification, in other words, the rewards you will receive immediately, then you will not do well on your math exam. Instead, you must make use of long term planning. Sure, I may not want to study today. It may be very annoying and I will miss my favorite TV show. But because you are planning long term, you’re not thinking only about today. You’re thinking about the results of your math exam. The desire to maximize the total future award is necessary for long term planning. We call the of future rewards. The return, we describe the return mathematically using the symbol symbology. Because it depends on future rewards, only it is time dependent. So we index it with a T. We can say the return at time T is the sum of rewards at time T plus one up to the terminal state a time a big T. Now, you might be wondering what happens if we have an Infinite Horizon MDP, a game that never ends, in this case, your return might be infinity. Therefore, we introduce a concept known as discounting. Discounting is used for infinitely long tasks, but it’s also used for episodic tasks as well. We introduce a discount factor called Gamma. Each future reward is weighted by Gamma to SunPower. Gamma is usually a number close to one like zero point nine zero point nine nine or zero point ninety nine. It’s a hyper parameter, so you’ll have to choose its value based on the performance of your agent. The idea is the further you go into the future, the harder it is to predict. Therefore, we care a little more about getting rewards now than we do later. Intuitively, this works just like money. I would rather receive one hundred dollars today than receive one hundred dollars ten years from now. And 10 years, 100 dollars will be worth much less than it is today due to interest. One important feature of the return which we will make use of throughout the rest of the section, is that it can be defined recursively, in other words, in terms of itself. Specifically, the return at time t is equal to the reward at time T plus one plus gamma times the return at times. Plus, why this may not seem like much more than a simple math substitution now, but you’ll see how it will become very useful later on.

6. Value Functions and the Bellman Equation

In this lecture, we are going to finally form the equation, which represents a generic reinforcement learning problem, and from this we can derive a solution. Let’s begin by discussing expected values. By the way, if you’ve never taken a probability course, then you’ve probably never heard of expected values, in which case, if you want to understand this, you want to take a course in probability to think about it. Simply the expected value is just the mean. So, for example, suppose I’ve measured the heights of one thousand students and I want to model this as a Gaussian distribution. I find that the average height is 70 inches with a standard deviation of four inches. In this case, the mean of the distribution is 70 and thus the expected value is 70. Let’s look at a coin as the next example. A coin has a binary outcome, if I flip a coin, I may get one or I may get zero. I suppose the probability of both one and zero is 50 percent in this case. What’s the expected value? Well, since it’s the mean, that means the expected value is zero point five. Sometimes this terminology confuses people. They ask how can the value? I expect to get the zero point five if the result can only be zero or one? It’s important to realize the expected value, despite its name, is not the value you expect to get. So if I flip a coin, I don’t actually expect to see the value. Zero point five. The expected value actually has a precise meaning for both continuous and discrete distributions, although for most of this section we will assume that we are working with discrete distributions. The idea is that the expected value is a weighted sum of all the possible values of a random variable where the weights are the probabilities of those values. As a simple example, we can return to our quaint example if we have a biased coin so that the probability of heads is zero point six, then the expected value becomes zero point six times one plus zero point four times zero, which is zero point six, because the probability of one is now a little higher than the probability of zero. The expected value also becomes a little higher. All right, so what’s the significance of expected values? Well, let’s remember that our reward and hence also our return, which is the sum of rewards, are random variables at a high level. What this means is that if I play a game one hundred times using the same policy in the same environment, I will get different rewards. This is because both the environment dynamics and my policy are probabilistic, and thus it makes sense to not think about a single return by itself. But the expected value of the return. I would like to maximize the sum of future rewards, but because the result is probabilistic, what I really want to do is maximize the expected sum of future rewards. They expected some of future awards has a special name and reinforcement learning, we call it the value function. I always mention that this is a very unfortunate name because the word value is such a generic word. Nonetheless, this is what it is called, because the return is the sum of future awards after we have arrived in some state. So, too is the value function. As you can see, it’s not just a plain expected value, but rather it’s conditioned on the fact that we arrived in the state as at times. Thus we denote this as vls the value of the state. S. As a side note, we sometimes refer to the return as simply the reward, even though the return should be a sum of rewards. So you always have to pay attention to the context. As you recall, I said that one of the most important things about the return is that it can be defined recursively the return at time t g of T is equal to the reward at time T plus one plus gamma times the return at time T plus one. And because of this, we can also define the value function recursively via is the expected value of the reward at time T plus gamma times V of as prime the value at the next date as prime. Now, you must be wondering, what is the purpose of all this? Are we just defining equations for the sake of it? Of course, the answer is no. Just keep in mind the high level picture. We are setting up a problem from which we can find a solution in order to find a solution. We must have a problem. And thus our job right now is to continue building up the problem so that it is well defined. The next step is to remember that the expected value is really just the weighted sum where the weights are the probabilities. So if we write it out in full, this is what we get. This is the sum over all possible actions, a. and the sum over all possible next states as prime and the sum over all possible rewards are inside the sum we have the probability of performing action a given that we are in state s multiplied by the probability of landing in the next state as prime and receiving the reward are. And then this is multiplied by the reward are plus gamma times V of as prime. In other words, it’s our plus gammer times have prime multiplied by the probability of actually getting the reward are and arriving in the state as prime. In other words, the expected value of the return. This is called the Bellmon equation. It is the centerpiece of all the subsequent solutions to reinforcement learning that we will discuss. It’s worth thinking about the bellman equation a little more notice that in order to come up with this equation, we only use the rules of mathematics, the rules of probability. It seems like just a regular old expected value. However, the implications are deep. It’s interesting to consider that the two probabilities here have given us an postponement. Argument never come from totally different physical processes. PI is our policy and it represents our Asian. So you can think of that as like an animal. The state transition probability represents our environment, so you can think of that as the world. So it’s interesting that while we can freely manipulate this equation mathematically, these two objects are actually completely distinct physically. At this point, you must be extremely tired, all we are doing is making our formulas progressively more complicated with no end in sight. But in fact, this is exactly where we want to be. Although this may seem complicated, I will demonstrate to you that, in fact, what we have arrived at is pretty simple. We can now define the first problem in reinforcement learning from which we can find a solution. Remember that in a reinforcement learning problem, you can have multiple policies. Some may be good, but some may be bad. How can we tell what is a good policy and what is a bad policy? Well, how about the value function? In other words, if I can find the value function for a given policy that tells me how good that policy is, we call the subscripts, perceives the value function. Given the policy pie. We call the problem of finding the value function, given a policy, the prediction problem. As promised, I said that we’ve arrived at something pretty simple. Let’s look at the bellmen equation again. These probabilities are a little deceptive. They have complicated symbols, but they are not complicated. Remember that the policy pie is just a function which we implement in our own code. We know this probability. Therefore, it’s not a problem. How about the state transition probability? Let’s assume this is known as well. This is entirely possible. Take, for example, the grid world environment. Now, what happens if we know these two probabilities? Surprisingly, this just becomes a system of linear equations, which you know how to solve from high school algebra. Suppose we have three states then if we simplify the submission from the bellman equation, we would arrive at something like this. So here the businesses are just constants which are derived from the probabilities which we’ve assumed that we know. From this, we can call a simple function like MPRDA Linowitz Utsav, and this would tell us via every one of us to invest three. In other words, if we know both our policy and the environment dynamics, we can solve for the value function using only linear algebra.

7. What does it mean to “learn”

In this lecture, I’m going to discuss what it means to learn in a reinforcement learning problem and reinforcement learning, there are two main types of tasks. Previously, we discuss what I call the prediction problem that is given a policy pie. Find the associated value function VVS. The second type of task is called the control problem. This means to find the optimal policy pie, which leads to the maximum VVS. In other words, the control problem is maximizing the sum of future rewards. You’ve already known since the beginning of the section that this is our goal. Now we can accurately define what this means and thus find a solution. First, we need a little bit more math and a few more symbols to accurately describe what we are doing. You already know about the value function A given a state VVS, to be more precise, we refer to this as the state value function. There is another quantity known as the action value function, where the value depends both on the state s and the action we denoted with the simple. Q As you can see, it has almost the same definition of yes, except that it’s also conditioned on the action. And so in the bellman equation, since A is given, we do not sum over the policy distribution. One interesting question to consider from a computer science perspective is how much space is required to store the value functions? Let’s consider the scenario where both states and actions are discrete and that we have a finite number of them. So let’s say we have big states and big AI actions. In this case, we can store the state value function in an array of size, a big —. But for the action value function, we’ll have to store it in a two dimensional array of size, big — times bigger, and thus the storage required for Q is quadratic, whereas the storage required for V is only linear. OK, so why do we need this concept of the action value? Well, this is going to help us find the optimal policy. Remember, some policies may be good and some policies may be bad. The optimal policy is the best policy, the one that maximizes the value. First, let’s just think about how to compare two different policies. We can say that policy one is better than policy. Two, if given Paswan is greater than Vivace given PITU for all the states as in the states face. So this helps us describe the relative good ness of different policies. From here, we can define the best policy and the best value function, the best value function is the value function for which there is no greater value function. It’s the max over all possible policies of Vivus given PI. So we’ll denote it with the symbol Vistar. Similarly, the best policy will be the Amax overall policies pie. So we call that pie star. So far, we haven’t needed to invoke the action value function, but let’s see how it’s related. First, the optimal action value is defined similarly to the optimal state value. It’s the max over all possible policies. Pae of Q given PI, we’ll call this Q Star. This must hold over all states and all actions. Furthermore, the optimal state value is equal to the max over all actions from the optimal action value. So from this, we can define the relationship between Q and V. The real value, no pun intended, of the action value function is this. Suppose we are playing some game and we would like to know what’s the best action to perform right now? Well, we have a dictionary telling us exactly what to do. All we have to do is find the Amax or Rikyu given a status. In other words, the best action to perform becomes a simple dictionary. Look-Up. As a sign, no notice how the optimal policy is not unique. It could be there. Multiple different policies lead to the same at best value function. In this case, it suffices to find just one of them. On the other hand, at the optimal value function is unique because if there are two different value functions, then logically one of them must be greater than the other one. Before moving on, let’s think about what a good first approach might be to actually finding an optimal policy. Remember, this is called the control problem. Suppose we are playing a game like Grid World or tic tac toe where our states based in action space are both finite. In this case, a naive search can solve this problem. First, let’s create a list of all possible policies that can exist, obviously, some may be bad and some may be good, but one or more of them can be defined to be the best. Then in a loop, we can test each policy, so first we call a function to find the value for the current policy. So that’s the evaluate function. Remember that as we discussed earlier, this is just a linear algebra problem. Then once we have the value function, we can compare it to our current best value function. If the new value function is better, then we make this the new best value function and make this policy the new best policy. When we are done looping through all the policies, we have found the optimal policy. In the next lecture, we’ll discuss these two functions here in greater detail. You may have noticed that we don’t yet really know how to implement them. What I would like you to know now is this. First, the evaluate function. Finding Vivus, given a policy is not all that hard. In fact, we already discussed how we would do this. If you knew both the policy distribution and the environment dynamics, it becomes a simple system of linear equations. In the next lecture, we’ll look at a more practical way of implementing this. Second, the other function enumerate all possible policies is conceptually simple but impractical as an exercise, you may want to try implementing this in code for a simple example like Grid World. In other words, while this method here appears to be nice and simple, it is not actually practical to do.

8. Solving the Bellman Equation with Reinforcement Learning (pt 1)

In this lecture, we are going to finally dive into some real reinforcement learning algorithms. Previously, we looked at some simple and naive solutions to reinforcement learning. Let’s recap what they were. First, we recall that there are two types of problems, the prediction problem and the control problem. The prediction problem means, given a policy, find violence. The control problem means find the best policy, which yields the maximum violence to solve the prediction problem. We noted that if we have the policy distribution and the state transition probabilities, it becomes a simple linear algebra problem. There are many algorithms and functions we can use to solve a system of linear equations for the control problem. We considered the naive method of simply looping through all the possible policies and finding which one is the best policy. Let’s consider now why both of these approaches are somewhat unrealistic. First, let’s consider the prediction problem. Previously, the solution required us to know both the policy distribution and the environment dynamics. But the reality is, if you imagine, say, any Atari game, we won’t know the environment dynamics. All we can really do, practically speaking, is play the game thousands of times. But usually the state’s space is so large that it would be infeasible to measure the state transition probabilities. And so unless you’re playing with a toy environment, you would not be told these probabilities either. Now, let’s consider the control problem, is it really possible to enumerate all possible policies? Consider if we have as big as possible states and big a possible actions in this case, the total number of possible policies is big A to the power base. In other words, this grows exponentially and hence it is not feasible to enumerate all possible policies for most practical problems. Furthermore, this would not even work in the case where the states space or action space is infinitely large. All right, so now that we’ve realized there is a problem, what is the solution? The trick to this is to remember the relationship between the expected value and the mean. The problem with the expected value is that it requires us to know the probability distribution of the random variable in question. But importantly, there is a way for us to estimate the mean or equivalently the expected value. This is called the sample mean. This is the basis for many scientific experiments. For example, if we want to test a drug, we don’t know the true values, but we can do an experiment on a set number of people and calculate the average values. The sample mean is simply the sum of all the samples we collect divided by the number of samples. The idea is that as an approaches infinity, this estimate will become more and more accurate. OK, so what does this have to do with reinforcement learning? Well, remember that the value function is simply the expected return. Therefore, using the sampling approach, what we can do is sample a set of returns for each state in the state space and take the average that will give us an estimate of the value of each state. You’ll notice that I’ve abused notation a little bit here by using different indices for the return g. When I say G of T, I mean a generic return G the return for the state at time t, but when I index G using I and s what I mean is this is a sample of the return. It’s the Iev sample return from the state s. Now, one obvious, but maybe not so obvious point, where do these samples come from? You may recall that when we’re numpty and we want a sample from, say, the standard normal, we can simply call a function and put out random doctorand in. But what does it mean to sample a return? Well, remember that when we play an episode, even if we use the exact same policy and the exact same environment, the result will be different. This is because both the policy and the environment dynamics are probabilistic. So just playing the game itself results in collecting samples. Using this concept, let’s now attempt to write some pseudocode to describe this algorithm, by the way, this approach is called the Monte Carlo approach, since this is a form of Monte Carlo sampling. First, we’re going to describe the prediction problem or in other words, given a policy, find the value function. Let’s just think about this at a high level. First, suppose we play one episode of a game. This consists of entering a series of states and corresponding rewards so we can call them S1 up to Estie and one up to Arte from this. How do we calculate the return of each state? Well, it’s helpful to actually go backwards, for example, G of Big T, a zero importantly, note that the return is only the sum of future rewards. Since a terminal state is the end of an episode, that means there are no future states, no future states means no future rewards. Therefore, the return of a terminal state is always zero and hence the value of a terminal state is also always zero. In any case, let’s continue. How do we calculate the return of the previous timestep? Well, by definition, G of Big T minus one is equal to R of Big T.. OK, the next one, how about G of big T minus two, it’s G of Big T minus two equal to R of Big T minus one plus gamma times R t. But importantly, remember that the return is recursive. So this is also equal to R of big T minus one plus gamma times G of Big T minus one. We can repeat this pattern. So G of Big T minus three is equal to R of Big T minus two plus gamma times G of Big T minus two. And of course this follows for any value of T, so we can say G of T is equal to R of T plus one plus gamma times GFT plus one. So you can keep repeating the same pattern until you calculate the return for each state. Practically speaking, here’s how you would do it in code first, you would play an episode using your policy and you get back a series of states and rewards. Next, we initialize a list of returns to an empty list and we initialize the return to zero then and this is the important part. You loop through the rewards in reverse inside the loop. We just use the recursive definition of G g equals to R plus gamma times G. Then we append to our list of returns. All right, so how do we put the above algorithm into pseudocode? Well, that’s simple. Everything we just did, now we just do it in a loop, do the thing we just did a several hundred or several thousand times then for each state, take the average return. First we assume we are given some policy. Then we instantiate an empty dictionary to store our sample returns. The key of this dictionary will be the state and the value will be a list of returns encountered throughout each episode. Then we enter a loop which goes on for a predetermined number of episodes. Inside the Loop, we first play one episode using the given policy. This returns us a list of states and corresponding rewards. From that, we can calculate the return for each state as previously described. Next, we loop through each state and return in corresponding order for each return we encounter. We append that to the list of returns for that state. This loop is how we collect our samples. Finally, when the first loop is complete, we are done collecting our samples. Now we can calculate the average return, which is by definition the value function. So we live through each of the items in our sample returns dictionary for each iteration. We get the state s and a list of sample returns G list inside the loop via us is simply assigned the sample mean of list. It’s important to realize that there are some complications with the algorithm as described first, because we’re only sampling, how do we ensure that we actually encounter every possible stay in the number of episodes we played? In fact, we cannot, although we may surmise that because we didn’t encounter a particular state, we don’t need to know its value because our policy does not allow us to go there. You could simply ignore the value of those states in any subsequent function you plug the value function into, but there is a better solution to understand it. We’re going to move on to the second problem, the problem of control or finding the best policy.

9. Solving the Bellman Equation with Reinforcement Learning (pt 2)

As a general pattern, it note that for the prediction problem, usually we are interested in finding Vivus, which is what we just did for the control problem, we are usually interested in working with the action value QSA. The reason for this is, if you recall, the Q function makes it easy to look up what action to take given a state s. We always take the Amax over all possible actions. We call this the greedy action and this way we maximize what we believe to be the sum of future rewards. So just keep this general pattern in mind for the prediction problem. We work with Vivus for the control problem. We work with QSA. In order to understand how to apply the Montecarlo approach to the control problem, we first have to understand the principle of policy, iteration and policy improvement. Let’s consider two basic facts. Number one, given a policy, we can use Montecarlo to evaluate the value function, whether that be the state value or the action value. We just saw that in the previous lecture. Number two, given an action value function, we can always choose based on this what we believe to be the best action given the current state. This is just the Amax over all possible actions A. Well, it turns out that these two facts are interdependent, given a policy, we can find its corresponding action value. And from that, we can take the AMAX to find a possibly better policy. But what if this policy is different from the original policy, then we can find the value function for this new policy and from that we can take the arguments again to find yet another new and possibly better, but at least as good policy. From there, we can find the value function again. So you see, this is just a loop where we go back and forth finding the value function a given a policy and improving the policy, given that value function. You can see that we have named these two steps, the act of finding the value function is called the evaluation step and the act of finding the best policy, given that value function is called the improvement step. And just to be clear, the reason why this is a loop is because they both change each other. So by doing step one, you change the policy and by doing step two, you change the value. It’s been proven, although we won’t discuss it here, that performing these steps leads to an monotonic improvement in the policy. Therefore, if we just keep doing this process, eventually we will end up at the optimal policy. So how can we apply this to Monte Carlo? Here’s a rough outline. First, we start by initializing QSA and a policy to both be random. Next, we enter a loop that goes for a predetermined number of episodes Inside the loop. We first evaluate the policy by finding QSA for the given policy. We call this the policy evaluation step. Once we’ve done that, we find a new policy where for each step we take the action to be the Amax overall actions for QSA with a given state. This is called the policy improvement step. This is pretty simple and not unlike the prediction problem, except for one small difference. Note that earlier when we discussed the evaluation problem, we discussed how to find a virus, but now we have to find a cure, say, in order to find QSA, where we evaluate our policy, we need to keep track of not only the states and rewards, but also the actions. So we’re going to record it triples of states actions and rewards. So as one a one on one as to who are two and so on. Here’s what the pseudocode for the evaluation step would look like. As you can see, this is pretty similar to calculating VLS, except when we saw the sample returns, we indexed the dictionary by both state and action. In the second part, Q is again a dictionary, but now the key is a state action tuple. Other than that, the process is exactly the same. We still calculate the sample mean of each list of returns for a given key. At this point, our solution works, but it’s not an ideal solution. Let’s think about why. First, if you recall, Vivus only stores big ass values, but QSA stores big ass times, big values. As you know, with Montecarlo sampling, the more samples you collect, the more accurate your answer becomes. When we use QSA, we have many more values to estimate and therefore we have to collect many more samples in order to get an accurate estimate. But there is a second problem, each policy evaluation step is a Montecarlo estimation, which means that we must calculate a large number of samples, but now we’ve nested this loop inside another loop. What’s the effect of this? Well, let’s say our policy iteration loop needs to run a 1000 times to find the optimal policy. Now, let’s say our inner loop needs to run 1000 times in order to get an accurate estimate of how many episodes do we have to play? The answer is 1000 times 1000, which is one million. As you can see, our data usage is not efficient and the number of times we need to play the game grows quite fast. A better solution is to use what’s called value iteration. At first, this approach may seem kind of wacky, but in fact it’s been shown to work. The idea is this for the policy evaluation step, instead of playing multiple episodes in order to get a good estimate of the value of our policy, we are only going to play one episode. After playing this one episode, we’ll get a series of states actions and rewards from which we can calculate the corresponding returns. From that, we can look through each state action and return and use the latest sample of the return to update of now you’ll notice that I’ve just put some pseudocode here, update queue of Asani, but I haven’t told you exactly how we’re going to do that. We’ll be discussing this shortly. The important thing to note right now is that will only keep a single running copy of Cuevas in the next. And final step is the same as before we update the policy by taking the Amax over queue for a given state over all actions. So what’s going on with this line here? How do we update you of given the old estimate of Cuba’s need for this, we need to take a short digression back to sample means and expected values yet again, as I’m sure you’re tired of me repeating by now, this is the expression for the sample mean we sum up all the samples and we divide by the total number of samples. The question we have to ask is, is this an efficient calculation? The answer is no, summing up and values is more than the more values we store, the more time it takes to calculate. This is not good. So the question becomes, is there a way to reduce the time it takes to calculate the sample? Mean every time I collect a new sample to see how this is done, let’s express the sample mean in terms of the previous sample mean and the latest sample. Before we move on, please make sure you can derive this yourself on paper in the derivation you see here, exboyfriend, that means the sample mean it using the first end samples, thus X bar and minus one means the sample mean using the first and minus one samples. The trick is we can turn the above expression into something that looks like and in fact is gradient descent, we can write it so it looks a little more familiar. The new estimate is equal to the old estimate, plus one over N times the sample minus the old estimate. And this scenario, the target is the latest sample we’ve collected, while the old estimate is our prediction, one over and is the learning rate. So this learning rate is one that decays over time by using such a learning rate. We get back exactly the sample mean. Remember, all we’ve done so far is basic algebra. Here’s what it would look like if we translated it into our Monte Carlo reinforcement learning code to update cue, the new estimate of Q of Santé is updated as the old estimate, plus one over N times, the latest return, minus the old estimate. Just note that the equals sign here means assignment. We’re not saying that both sides are equal. The left side is the new estimate and the right side holds the old estimate. We just haven’t bothered to subscript them. However, we are not done yet because we are going to make yet another modification to this. Remember that in our latest value iteration loop, we are updating the same Q dictionary using different policies. This policy is being updated on each step and therefore the samples which we are using to calculate Q of Santé are not coming from the same distribution. In this case, we don’t want to use exactly the sample mean. Intuitively, the oldest samples come from the oldest policies. They don’t really matter that much. The newest samples come from the newest policies and they matter more. The key is to use a constant learning rate instead of one that decays over time. Doing so gives us what is called the exponentially decaying average. The idea is when you take the conventional sample, mean each sample is weighted equally. As we said, we don’t want that because old values come from old policies. And so they come from a different distribution than the one we are now interested in. On the other hand, the exponentially decaying average weights, each sample and an exponentially decaying fashion. The most recent sample matters most, and the weights for each sample decay exponentially as you go backwards in the order they were collected. So now we can express our value iteration code again, but this time we can fill in this little missing detail about how to update QSA. So this is the exact same pseudocode as before, except I’ve replaced the middle block of code with the actual update we can use to update you. Unfortunately, at this point, there is a subtle but important detail we have yet to discuss, so we are not quite done specifying our algorithm, but we’ve laid out most of the groundwork.

7. VIP Reinforcement Learning for Algorithmic Trading

1. Trend-Following Strategy with Reinforcement Learning API

In this lecture, we are going to look at how to reimplement the trend following strategy that we looked at in one of the previous sections, but this time we’re going to design the program so that it fits in with the reinforcement learning paradigm that we just learned about. The goal of this is to eventually get secure learning, but to do it in steps in this first step will introduce you to several important components, such as the environment, states, actions and rewards. Furthermore, I think that this design makes it more clear what the timing is. If you recall, when we implemented a trend following earlier, we had to shift the returns, which probably seems kind of sketchy at the time. But since reinforcement learning inherently works over discrete time steps, we can take a much more natural approach. There will be no more. We are shifting that has to take place. It will seem less like manipulating a data frame and more like we are controlling an actual robot that makes trades within an external environment. I think it will be useful to walk through a data frame, a step by step and consider what the state’s actions and rewards actually are, we can begin by assuming that we have a data frame of espie close prices along with the fastest man, slowest among which we already know how to compute. We also have the daily log returns, which are now not shifted. That is the return that we see on each row is the return for that day. OK, so let’s suppose we are at some arbitrary row at time t let’s also suppose that we keep track of whether or not we are currently invested and holding stock or not invested and holding cash. OK, so let’s suppose that we are currently not invested. Then we noticed that today the faster sum is greater than the slow estimate since we were previously not invested. This means that the fast the same has crossed the slow Esmay from below. So now it’s time to buy. Of course if we buy it, today’s closing price, that is at the end of today, we don’t get today’s returns. However, it does mean that we will be invested tomorrow at time T plus one this action and brings us to the next day, which is tomorrow. As fast as a man, slower some day at time T plus one. Furthermore, we also receive the corresponding return, which we will be using as the reward at time T plus one. OK, so this is exactly an MDP transition, we were in state as of T and we performed the of T, this brought us to the next state, Osiecki plus one, and we received the corresponding reward are at times E plus one. OK, so let’s dig deeper into the actions we know that the state is a vector of size to containing the fastest O’May and the slow Esmé, may we know that the reward is just the return, but the actions are a bit less trivial. Previously, we thought of having a column called is invested, which was either true or false. However, this is an an action persay, but more like a state of being. The action in actuality is either buying, selling or doing nothing. On the previous slide, we said that if we are not invested and we see that fast sum is greater than slow Esmay, that an action should be to buy. Let’s try to think of what rules we can come up with to figure out when we should sell or when we should do nothing. So let’s consider when we would want to do nothing. If we are already invested and we find that fast US sum is greater than slower Samay, then we should simply remain invested. In other words, we should do nothing if we are already not invested and we find that faster. Simay is less than slow, Esmé. Then again, we should remain not invested. The only other case is when we are invested and we find that faster sum is less than slow. Esmé, in this case we should sell. We should make sure that our previous logic makes sense, why does it work if we only look at today’s fastest man slower day, along with keeping track of whether or not we are invested? As you recall, what we really want to know is whether faster, some cross slower, some from above or below. This is usually how the trend following strategy is defined, but this requires keeping track of the previous fastest man, slow Esmé. The best way to understand this is with a picture. So we are interested in the event where faster some cross is slower. Samay But what happens in between these events? Well, it’s either the case that fast same is bigger than slower Sam, or vice versa. If we find that faster, Sam is bigger than slower SAMAY it must have crossed slower Sam from below at some point. Therefore, we should be invested during that time. So if we are already invested and we see that faster, Sam is bigger, then there’s nothing to do. We should just remain invested. But if we find that faster, Sam is smaller, then that means faster. Sam must have crossed from above and therefore we should sell. The same logic applies if you reverse the science, so if you find that faster Samay is smaller, then it must have crossed slower Samay from above at some point. So either you’re going to sell if you were invested or you’re just going to remain on invested. Now, of course, at this point, you realize this actually just depends only on whether or not fascism is bigger than slow. Esmay However, I think it’s more clear and more natural to work through the actions and the timing. Now that you understand the logic behind how the states actions and rewards are related, we can talk about how we are going to implement this in code at a high level. We’re going to have two central objects. One is the agent and one is the environment. Instead of starting with what goes inside these objects, we should start by thinking about how they should behave. That is to say, what is their API? Note that for the environment, I’ve taken inspiration from a library called Open API Jim, which is a popular collection of common A.I. environments. So if you ever work with Open Jim in the future or you’ve already worked with Open Ajam Jim, this should make things easier. So we’re going to start by instantiating two objects, one of Type N.V. and one of type agent envy takes in one argument, which is either the train data frame or the test data frame. Let’s consider how we will use these two objects to play one episode of our game. So the first thing we always do is call Invidia Reset. This puts us into the initial state for the episode. It also returns this state. Next, we set our done fly equal to false and initialize the total reward to zero. Next, we enter a loop that goes on forever until Dunn becomes true. Inside the loop, we call Agent Dot Act passing in the current state. This gives us the action that our agent decides to do for this state. Next, we call each step passing in the action that we just obtained. This returns the next day, the corresponding reward and a flag telling us whether or not we’re done, that is whether or not this next day is a terminal stay. Next, we add a reward to the total reward. Optionally, we can pass all this information into Agent Gautrain in order to update our agents logic, for example, with Q Learning with trend following, there will be no such logic. Next, we assign the next date to the current state variable for the next iteration of the loop. OK, so this small loop tells us how we expect the Asian and the environment to behave. The environment has two functions reset and step. The agent also has to function, act and train. Next, let’s think about how the environment will actually work. Keep in mind that this is simplified pseudocode, so when we see the actual code, it may be a bit more complex. So, as mentioned, the two important functions are reset and step inside the reset function. You can see that we’re doing two things. One, we’re setting an instance variable called current index to zero. Two, we’re indexing an instance variable called states at current index, which we just set to zero. Thus, states contain all the states of the episode, which, as you can gather, are the columns of the data frame corresponding to faster Simay and Slow Esmay. Next, we have the step function, this is responsible for completing one transition in our environment. It should return the next day the corresponding reward and whether or not we are done. So the first thing we do in this function is increment current index by one. Next, we check the action. If the action is by then, we set an instance variable called is invested to true. If the action is sell, then we set it to false. Otherwise we do nothing. Next, we compute the reward, if is invested is true, then we should return the log return at the timestep corresponding to current index. If is invested as false, then we are just holding cash and so our return would be zero. Next, we get the next date by indexing self-taught states by current index. We also check whether or not we’re done. Done will be true if current index is equal to the length of the states minus one. This simply means we have run out of data to walk through. Finally, we return next state reward and on. Next, let’s consider our agent, since trend following doesn’t do any training, we only have a single function called Act, this takes in a single argument, which is the state you can think of the state as a list or a tuple of length to where the first element contains the fastest samay and the second element contains the slow Esmay. Therefore, if state zero is bigger than, say, one, and we are not invested, we set is invested to true and we return the action by. So notice how both the Asian and the environment have their own instance variables called is invested. Next, we check the cell case, if state zero is less than state one and we are invested, then we said is invested to false and we return the action. So if neither of these two cases are met, then we return a do nothing. OK, so hopefully that gave you a good idea of the basic setup for a reinforcement learning program, you know how the agent and the environment will interact and you also know how the agent and environment work. Note that in some reinforcement learning tasks, the environment will exist outside of your program. For example, in open a gym, all the environments are part of that library. You only write the code to interact with it in the physical world environments such as flying a drone or controlling a robotic arm. The environment is the world itself.

2. Trend-Following Strategy Revisited (Code)

In this lecture, we are going to reimplement the trend following strategy, this time we’re going to use a reinforcement learning API where we have explicit environment and agent objects which interact with each other. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. So a lot of this stuff is going to start out the same as before, but just in case you spent a long time learning about reinforcement learning, I’ll go through everything again as a review. So we’ll start by downloading the Aspies CSFI. Since this is a trend following, we only need the daily clothes Brice’s. Next, we’re going to import pendas, non-pay and matplotlib. Next, we’re going to Reena Naqvi using pedigreed CSFI. Next, we’re going to calculate fast Azmeh and Slow USMA will also create a variable called FTS, which will contain the column names of these moving averages. Next, we calculate the log return by taking the log of the close prices and then the difference. Next, we split our data into train and test with the last 1000 points for the test said. Next, we implement the end of class, the first function we have is the constructor, the constructor takes in one argument, which is a data frame. This will either be the train or test data frames we just created. Inside this function, we set D.F. to an instance variable. We set PN the length of D.F., which we can use to determine when we reach the end of the data frame. Next, we set current index to zero. This will keep track of our current position in the data frame. Next, we define the action space, which will be the integers zero one and two corresponding to buy, sell and hold. Next, we said our invested flag to grow this will either be one or zero corresponding to true and false. Next, we set the state’s instance variable. This will be the data frame columns specified by the feature list we defined earlier. We’ll also convert this to a Nampara for more convenient indexing. Next, we set the rewards instance variable. This will be the log return column from the data frame. Also convert it to an umpire. Finally, we’ll also keep track of the buy and hold return. This will be accumulated in an instance variable called Total Buy and Hold by the end of the episode. It should be equal to the sum of all the log returns that could have been rewarded by the environment. Next, we have the reset function inside this function, we reset the current index to zero, then we set the buy and hold Total Jazeerah. We also reset the invested flag does zero. Finally, we return the initial step, which can be obtained by indexing self-taught states at location zero. Next, we have the step function, this takes in one argument, which is the action that the agent performs, recall that the step function should return three things the next day, the reward and the done flag. So all the work we are about to do is to compute these three things. We start by incrementing the current index. Conceptually, this is bringing us to the next day as a sanity check. We can check whether or not current index is greater than or equal to N, since it should never reach this point. We raise an exception if it does. Next, we check the action, if the action is by, we said invest it to one or true, if the action is sell, we set invested zero or falls. Next, we compute the reward, if we are invested, then we return the reward at the current index, otherwise the reward is zero because we are not holding any stock. Next, we obtain the next day, this is simply self-taught states, index by current index. Next, we add the current log return to the total buy and hold counter. Finally, we set the Dunn flag. This should be true whenever current index is equal to and minus one. Lastly, we return the three things we promised we would the next day the reward and the Dunn flag. Next, we defined the Asian class. First we have the constructor, which does not take in any arguments inside the constructor. We set one instance variable is invested to false. Next, we have the function at this takes and one argument, which is the state inside the function, we do a sanity check to ensure that the state has length to the state should be a vector containing the fastest slow Ismay’s. Next, we determine what the action should be. So afast is greater than slow and we are currently not invested. Then is invested, becomes true and we return zero, which means by if fast is less than slow and we are currently invested. Then we said is invested to false and we return to one, which means so if neither of these cases are met we return to which means do nothing. Next, we have the play episode function, this function takes in two arguments, which are the agent object and the N.V. object inside the function. We start by calling Ian Arissa. Next, we initialize Dunn to false. Next, we initialize total reward to zero. This function will accumulate the total reward for the episode and then return it at the end. We also make sure to set Agent Dot is invested to false, so we begin each episode uninvested. Next, we enter a loop that finishes when Dunn becomes true inside the loop. We begin by calling Agent Dot Act to obtain the action for this day. Next, we call Ian Verrat step passing. In the action, we obtain the next state reward and done flag. Next, we accumulate the reward in total reward. Next, we update the state variable to the next day so that on the next iteration of the loop it represents the current state. When we are finished, we return the total reward. Next, all we have to do is make use of all the hard work we have just done. We start by creating two environments, one for train and one for test. Next, we instantiate our agent. Next, we call play one episode using our agent on a Train N.V.. Next, we call play one episode again, this time using test envy. Next, we print out the train reward and also the buy and hold total for the train said. We get about zero point four three and zero point five nine seven, which correspond to what we got before. Next, we print out the test reward and the buy and hold total for the test said. We get about zero point zero eight eight eight nine and zero point one nine three, again, a corresponding to what we got before. OK, so I hope that this lecture helped to solidify a few concepts. Firstly, I hope it made our code from earlier make more sense when we first encountered trend following. It was more like manipulating a data frame in this lecture. We actually got to implement an agent and an environment and we could see them interact in a more realistic way. Secondly, you learned about some very crucial components for reinforcement learning, including the environment, Asian states, actions and rewards. Knowing what you know now, the key to learning scripts will seem like a small adjustment to what we’ve done so far.

3. Q-Learning in an Algorithmic Trading Context

In this lecture, we are going to do a short primer on cloning in a financial engineering context, you can imagine this lecture as a shortcut to cure learning. If you couldn’t bear to sit through two hours of pure reinforcement learning theory, obviously you should still do that. But this lecture might give you a different perspective. OK, so unlike the previous trend following code in this lecture, our Asian will actually learn and use its experience to improve future outcomes. So how can to do this? We start by introducing a concept called The Table. The cuttable is indexed by two things the state and the action. You can imagine that if we put the states along the rows and the actions along the columns, we would get exactly what looks like a table. Hence why we call this a table. You’ve already seen that actions can be represented by the integers zero, one and two, so that makes sense. Action zero corresponds to column zero. Action one corresponds to column one and so forth. But what about the states? As far as we know, the states are not currently encoded as integers, so how can they index the rows of a table? This will be a topic for the next lecture, but for now, let’s just assume that we will be using a table and exactly this form. OK, so what does Kieu actually represent? The cuttable represents the expected return for doing the action a while in status. Note that you should not get reinforcement learning returns confused with financial returns. When I say return in this context, it means the sum of future rewards. So Kufor status and Action A represents the sum of future rewards. I will expect to get in the next timestep, the timestep after that and so on. Using this definition, it becomes easy to decide what the optimal action is. Obviously, I want to maximize my future rewards. Therefore I should pick the action that gives me the maximum. Q This might seem like circular reasoning, but let’s assume that HEW really does contain these expected returns. Then it should be clear that if I find myself in some status, I can use the cuttable to determine what the best possible action is. I simply need to search through all the possible actions to find the maximum. Q That is my policy should be to take the Amax over QSA for all possible actions. So that takes care of how we should use the cuttable, assuming it contains the right values. But how do we make sure the cuttable contains the right values in the first place? For that we need Cullinane. Since you’ve all been through the Times series section of this chorus, you’ll recognize that learning is nothing but the exponentially weighted moving average. Firstly, we know that the return is recursive, the return of G at time T is equal to the reward at times plus one plus the return G at time, T plus one. Similarly, we can estimate Q at the current state by taking one step, receiving the reward from the next state and using the max over. Q In the next day we take the max, assuming that we will follow a policy and perform the optimal action on each step. So this is like taking the expected value of both sides. We’ll call this a better estimate of Q and will assign it the variable Y, we’ll call it Y because it’s a target. So how do we update Q? Q is nothing but the exponentially weighted moving average using Y as the so-called new sample. So you can imagine this from the context of calculating the sample mean to calculate the sample mean we take the weighted some of the new sample and the old estimate. Eventually we know that this estimate will approach the Trumaine similarly, including we can think of Y as our new sample and the Q on the right as our old estimate. We update. Q by mixing these two into a new and hopefully improved estimate. Using basic algebra, we can rearrange this into a more typical form where QSA appears by itself and the error between Y and QSA appears beside Alpha from my machine learning students. You may recognize this as gradient descent or gradient descent. Note that this is the version we will see in the code since it’s closer to how Q Learning is usually presented. OK, so that covers the two aspects of Cullinane, how do we use cue to take actions and how do we update you so that our agent learns to perform optimal actions? If you recall earlier we said that our agent object needs two functions act and train. Knowing what we know so far, we can now write the pseudocode for our cue learning agent. OK, so let’s start with the act function, as you know, we don’t just take the greedy action all the time, we have a small probability, Epsilon, of performing a random action. This helps us balance the explore exploit dilemma. We want to exploit what we think is the best action, but we also want to explore and collect more data so that our estimates improve. So in this function, we’re going to generate a random number between zero and one. If this random number is less than Epsilon, we simply sample a random action from the action space. Otherwise we select the action, which is the arc over. Q With a given status for all possible actions a. Next, let’s consider the train function, this function will take in an entire MDP transition as arguments so we have state action reward next state and done flag if done is equal to true. That means next state is a terminal state. We know that the value of terminal states is zero since there can be no future rewards. Therefore, the target way is just the reward itself. Otherwise, we use the coloring formula to say that why is the reward plus the discount factor GAMMER times the max of Q given the next state over all possible actions, then we update our cuttable using the exponentially weighted moving average. So this lecture was a very brief introduction to keep learning, making use of the concepts we learned about earlier in the course. We now know how to use the table so that we can perform optimal actions. We also know how to update the cue table so that the values it contains are actually useful. These correspond to the two important functions of our Asian class, which are act and train.

4. Representing States

In this lecture, we are going to discuss one key item, which I purposely left out of the previous lecture, and that is how do we represent the states? First of all, we haven’t yet discussed what kind of data we’re going to use. Well, to reiterate, one of the main themes of this course, your data can be anything in the world. It’s your job to pick something useful and tested in this course for the sake of simplicity and comparison. We’re going to use the same data that we used in our machine learning scripts. That is, we’re going to use the previous day’s returns for various stocks in the S&P. OK, but now we have a problem. Returns are continuous, valued variables, there are infinite numbers of them. In order to have a cuttable, our states must be discrete. And in order for our cuttable to fit into memory, we must have a finite number of states. So how can we tackle this issue? One simple solution is to use beaning. The basic idea is this. Let’s say we have three bins. We can say everything from minus infinity to zero goes into been one. Everything from zero to one goes into beento and everything from one to plus infinity goes into bin three. So that’s beaning. You can take an infinite number of values and squash them down into a finite number of discrete bins. The next question you have to consider is how do you define your. Obviously, the previous scheme isn’t that useful since most returns are very close to zero. So suppose we take the maximum value and the minimum value and we split that up into, say, 10 equal parts? This is better, but it’s still not ideal. As you recall, returns have distributions. Some values are very likely. Usually the ones around zero and some values are very unlikely. You don’t want a special bin to take care of some extreme return that you only saw once. Conversely, you don’t want to throw 90 percent of your returns all into the same bin. Then your model will have trouble learning anything useful. You don’t want everything you see to be represented by the same state. So what we would like to do is define our bins according to frequency. That is, areas with higher frequency should have smaller bins and areas with lower frequency should have larger bins. The bins on the left and right should extend to infinity. Luckily, there is an equivalent, an easier way of saying this. It’s simply that each bin should contain an equal number of data points. So that’s the theory, but how do we implement this in code? Let’s start with the easy part and then we’ll get to the hard part. If we know the boundaries of our bins and we have some value, how do we determine which bin it belongs to Numpty as a function that does exactly this called digitise? Here’s an example from Numpties documentation. We can see that the bins variable defines the boundaries of the bins, they are zero one, a two point five, four and 10. When we pass in X, we get back the bins one, four, three and two corresponding to the bins for each of the values in X. This makes sense, zero point two is between zero and one, therefore it goes into been one six point four is between four and 10, therefore it goes into bin four. Now, what this example does not show is what happens when you go outside of the boundaries, which is definitely possible in our case. You’re encouraged to try this on your own to verify the results. But here’s what would happen if I pass in minus one, which is less than the smallest bin boundary, I get assigned to bin zero. If I pass 9/11, which is greater than the largest bin boundary, I get assigned to bin five. I think this makes perfect sense, since the full set of beans are then zero, one, two, three, four or five. OK, so next, let’s discuss the hard part, which is how do we find the boundaries in the first place? Conceptually, this is pretty easy. All we have to do is sort the returns. Now we have an array of sorted numbers from smallest, the largest. Since this is an array, we can look for equally spaced elements in the array. So as an example, let’s say our array contains 100 elements and we want to split it using 10 binn batteries. Since we want our model to handle values that can stretch to infinity, the boundaries of Harbin’s should be centered rather than lined up with the smallest and largest values. So, for example, I would have a boundary of five, fifteen, twenty five, thirty five and so on, up to ninety five. In this way, the first five values will belong to the same bin as, say, minus infinity, and the last five values will belong to the same Binnaz plus infinity. One last detail I want to discuss in this lecture is how we will represent the cuttable since we’ll be looking at the returns from multiple stocks, our state will actually be a tuple of integers rather than just a single integer. Luckily, Python has the perfect data structure for this, which is the dictionary. As long as your object is immutable, you can use that object as a dictionary key. In our case, the state will be a tuple of multiple integers and the action will be an integer. So we’ll actually have a nested tuple of integers. This is immutable, which means we can use such an object as a dictionary key. So conceptually, the table is a table, but in our code, it will be an array indexed by objects otherwise known as a dictionary. OK, so that was a very brief lecture on how we will represent our states in the upcoming court lecture note that this is pretty complicated. So it’s understandable if you don’t understand what we just talked about, especially since there wasn’t much context. However, I think it’s useful for you to see everything twice. And so you should feel free to go back and forth between this lecture and the next lecture until you understand all the details. Better yet, as I always say to my students, if you don’t like my solution, create your own. I’m only here to teach you the concepts and to give you an example implementation. It’s not meant to be the fastest implementation nor the most Pythonic implementation. It’s the implementation that makes sense to me and hopefully it makes sense to you. I think doing things the way I just described helps you to think through all the steps clearly and unambiguously. Your goal should always be to implement things in a way that’s most intuitive for you.

5. Q-Learning for Algorithmic Trading in Code

In this lecture, we are going to finally implement our learning trader since the script will use the same data as our machine learning script. We’ll have to download the full S&P data set. Next, we import Pendas, Numpty, matplotlib and or Tool’s, you’ll see how we make use of Aitor tools shortly. Next, we load in our data frame using to read CSFI, since you’ve already seen how we process this data frame. We’ll go through this more quickly. So first we drop any rows that have all missing values since these rows are irrelevant. Next, we drop any columns with at least one missing value since we don’t want to bother with forward filling or backfilling. Next, we generate a new data frame containing all log returns for each of the S&P components. Next, we split the data into train and test using the returns data frame. Next, we specify that our input features will be Apple, Microsoft and Amazon, by the way. It’s worth noting that because we are using binning, the state’s base will grow exponentially. If each column has 10 bins and we have three columns, then there are ten times ten times ten unique bins in total. If we have these features, then we would have 10 to the power bins. This is exactly exponential growth. And so using this method, the time and space requirements of your program will grow quite fast. Next, we have the end of class. This is mostly the same as it was before. So we’ll go through this quickly. The main difference is that inside the constructor, the rewards are now the spy column of the data free. The features are also different, but the code is still the same since our features are specified by a variable called FTS. Next, we have the state map reclass, this is probably the most complex piece of code in the script, it relies heavily on the theory lecture that came before. So I’ll mention again that if you don’t quite get this, then you’ll want to go back to the theory lecture to make sure you understand everything we discussed. So the purpose of this object is to transform a given continuous vector of states to the correct Binz. The constructor is responsible for defining the bin boundaries, it takes in an object and a binz specifying the number of bin boundaries and the end samples. The basic idea for the constructor is this. As you know, we want to have a bunch of sample values for each of the state variables so that we can estimate their frequency in order to collect the samples. We’re going to run through the environment many times, performing only random actions. After we’ve collected these samples, we can sort them and define the boundaries as we described previously. So first, I’m going to create an empty list called states. This is where we will store the states that we collect. Next, we said done to false and we call Ian varicella. Next, we set an instance variable D to be the length of the state vector. This will be the number of sets of bins that we need to create. Next, we append the initial Staib to our list of states. Next, we enter a loop that iterates and samples times inside the loop. We choose a random action from the environment’s action space by calling NNPA random dot choice. Next, we call Ian Verrat step passing in our random action. This gives us the next date as to which we also append to our list of states. Next, we check whether or not we are done. If we are, then we reset the environment and start again. Next, now that we’ve finished collecting states, we cast it to a new era for easy indexing by Rowand column. Next, we need to create the bin boundaries for each column of data. So we start by creating an instance variable called Selvig Bins. This will be an empty list and will update this with the bin boundaries for each dimension. Next, we live through each of the dimensions inside the loop. We thought that the fourth column of our state’s array will call this variable column. Next, we want to find the boundaries for this column of samples. So we start by initializing an empty list called current bin. This will store all the boundaries. Next, we do a loop in bins times. Note that this is a bit of an inaccurate name. It’s not actually the number of bins, but rather the number of bin boundaries. Inside the loop. We find the boundary by taking end samples, dividing it by and Obinze and multiplying it by K plus zero point five. This ensures that we sent to the bin boundaries as discussed in the three lecture. You might want to plug in some numbers to make sure that this roughly makes sense. For example, let’s suppose we use the numbers from the theory lecture so we have 100 samples and 10 bins. If K equals zero, then we get one hundred divided by ten, which is ten. We multiply ten by zero point five, which gives us five. So our first boundary is five, which is correct. Our second boundary is ten times one point five, which is fifteen, which is correct. Our last boundary is ten times nine point five, which is ninety five, which is correct. OK, so at the end of this loop we’ll have all the boundaries stored in the list current bin. Then we simply append the current bin to self Dobbins. Next, we have the transform function. This is the function we call when we want to convert a continuous state vector into the corresponding been tuple. OK, so this function takes in one argument, which is the continuous stay. Next, we create an array of zeros of length D called X. This is where we will store the bin values. Next, we loop through each of the dimensions. Inside the loop we call NP digitize on the state value using the forbidden boundaries. We assign this to X at location D. Finally we cast X to a tuple and return it. Next, we have a function called All Possible States, it may not be clear why we need this yet, but recall that when we create our cuttable, it is updated iteratively. That is, the new value always depends on the old value. Therefore, each inputs AQ must have some initial value. Therefore, we must provide an initial value for all the states and therefore we need to know what all the states are. OK, so to do this, basically the secret sauce is to use the tools that products function, you’re encouraged to do some simple examples on your own so you can understand how this works. Basically, we need to create a list. Each element of this list must itself be a list. Each of these lists should contain all the possible bins for that dimension. As you recall, these count up from zero to one plus the number of been boundaries. So the code here is a bit verbose, but if you look at it carefully, it should make sense. The number of bins is the length of the bins list. And then we had one we call the range function to go from zero up to that number. Then we cast it to a list to convert it to a list. Finally, we call it or tools dot product to get all the possible combinations of the elements of these lists, as mentioned previously, the best way to understand this is to simply try a few examples in your console. OK, so next, we have the agent class, this is yet another chunky bit of code, as you can imagine, this will be quite a bit more complicated than our trend following agent, which doesn’t learn at all. OK, so first we have the constructor. This takes in two arguments, the size and the state mapper object. We start by assigning a bunch of instance variables. So these are action size, discount factor, Gamma Epsilon learning rate and state mapper. You already know what all these things are, so I won’t explain them. Our next job is to initialize the cuttable, as mentioned previously, the cuttable is updated iteratively and therefore each of the values must start somewhere. We’ll start by initializing Q to an empty dictionary. Next, we live through all possible states by calling state mapper dot all possible states. Inside this loop, we cast the state s to a tuple. This is because the error tools returns a list, but lists are not immutable. Next, we live through each action as well. This is because Q is indexed by state and action, so we need all possible combinations of these. OK, and next, we assign the Q value for this state and this action to be a random number drawn from the standard normal. All right, so that’s the constructor. Next, we have the function, this is still just epsilon greedy, but a little more complicated since we need to search our cuttable. So first we generate a random number between zero and one if this random number is less than Epsilon. We return a random action. Otherwise we continue. We then use our state map here and we call the transform function to convert the state into its been representation. Next, we look at our table for the state and all possible actions. We’ll put this into a list called Act Values. Next, we take the Amax of actor values to return the optimal action for the state. Next, we have the train function. The goal of this function is to implement the key learning formula that you learned about earlier inside this function. We first transform state and next state to their been representation. We’ll call these essense two. Next, we check whether or not as to is a terminal state by looking at the done flag. If it is, then the target is equal to the reward, otherwise we grab all the possible values for the next state as to over all possible actions, we’ll call this act values. Next, we create our TRD target, which is the reward, plus gamma times the max over values. Finally, we apply our learning formula to update IQ for the given state and the given action. Next, we have the play one episode function, this is mostly the same as earlier, except now we have one extra argument called is train. As expected, our model will train only on the train set, but not on the test set. So, again, this is basically the same as before. The only extra line is inside the loop. Where we check is train is train is true. Then we call Agent Train, which hopefully makes sense. All right, so next, all that’s left to do is make use of everything we just created. So let’s start by setting some episodes to five hundred. This is the number of episodes that will be used to train our agents. Next, we create two environments, one for train and one for test. Next, we set the action size to the length of the action space. We also instantiate the state mapper and the agent. Next, we create two empty arrays to store the total reward we receive from each episode for both training and test. Next, we do a loop, numb episodes, times Inside the Loop. We first play one episode with the train environment we set is Train the crew. Since we want our agent to learn during this process, we get back the reward are and we assign this to our train rewards array. Next, we run our agent on the test set before we do this, we’re going to set Epsilon to zero so that our agent does not explore during testing. We start by saving the current epsilon in a variable called temp epsilon. Next, we set agent Dot Epsilon to zero. Next, we play one episode with the test. Set setting is Treinta Falls. After that’s complete, we reassign a temp epsilon to agent Dot Epsilon. Finally, we save the reward to our test rewards array. At the end of this loop, we print out the episode number and the reward for both training and test. OK, so let’s run this. OK, so the last step is the plot, both the train and test reward’s. OK, so as you can see, our model does pretty well, obviously, it does much better on the train set, since that’s what our agent learned on the note, that since there is some randomness in this algorithm, you’re going to get a different result. Each time you run this, however, you’ll notice that pretty much every time. Q Learning beats buy and hold on the test set. You’ll also notice that as the agent learns, the variability in the test, rewards seems to go down. This is a good thing because we can be more certain that good performance is due to doing something intelligent rather than random luck.

8. VIP Statistical Factor Models and Unsupervised Machine Learning

1. Statistical Factor Models (Beginner)

In this lecture, we are going to study statistical factory models, statistical factor models are an example of unsupervised learning. This is very different from the models we discussed earlier in the course. Earlier in the course, we looked at models such as Arima, where the goal was to make an accurate prediction. We knew that the predictions were accurate when they were close to the targets. One way to picture this is you have your data X and you have your targets. Why your model outputs Y hat and your goal is to find the parameters of your model so that Y hat is close to Y, so you can think of the box that goes from X to Y as the real world phenomena that maps the input to the output. Your goal is to model this real world phenomena such that your models output is close to what happens in the real world. OK, so we call this supervised learning. But unsupervised learning is different again, we can use a picture in unsupervised learning, we say that it’s unsupervised because we are not given any targets. There is no supervision or a teacher in that sense. All we have is X, but we are still trying to make a prediction in our picture. There is some hidden variables that we are trying to predict. The way that Xanax or related is this Xanax are related through real world phenomena, but instead of X going to Z, Z goes to acts. So before we had X going to Y, so X was the starting point. In this case, X is not the starting point, but rather the endpoint. Z is the starting point. We call Z the head and cause or the latent factor because there’s something that we can’t see that is causing the data that we observe. An example of this is when you go to the doctor, you go to the doctor because you have symptoms which are things that you can observe, but you don’t know the hidden cause. So the cause is hidden. You hope that the doctor will choose the right head and cause so that you can be treated appropriately. So what happens when we build a model to predict Z for the sake of this picture, we’ll call the predictions and see how well it’s the same idea as with supervised learning. We want Sehat to be close to Z. That is, we want to build a model that accurately reflects the real world phenomena. Of course, the problem is we never actually get to observe Z because it’s hidden. The only data we observe is X. In any case, this is just a very high level description of unsupervised learning in the rest of his lecture. We will look at more specific methods to actually build models that take us from A to Z, have. The specific technique we’ll use in this lecture is called Principal Components Analysis. And just as a side note, since we never actually get to observe Z, we’re going to conflate Z with Zayat and just call model predictions a Z without any how it should be understood that these are predictions. Let’s first motivate this problem by starting with the cap in the CAPM says that all stock returns are linearly related to a single factor, which is the market return. Beyond this, there may be some noise which makes it so that the returns on any particular stock do not exactly track the market. But as we stated earlier, the capture is what we call a single factor model. There are other models where stock returns can depend on multiple factors. One example of that is the Fama and French three factor model. And in this lecture we are going to study statistical factor models in these models. We assume that we can never directly observe the factors and instead we treat them as hidden causes. Let’s look more closely at the form of this factor model, the factor model can be stated as follows. It says that the return for Starky at time t is a linear combination of different factors at time. T we use the letter Z for these factors. So we have a Z, one is A to Z three all the way up to Zek. For now we’ll call the coefficients for these factors beta so that they correspond to the financial data we discussed earlier. OK, so hopefully this is pretty intuitive, it’s exactly like the CAPM, except that we have more factors and also we do not observe these factors directly. Let’s think intuitively about how we can find these factors. Consider this scatterplot of two highly correlated variables, height and weight. Generally speaking, the taller you are, the more you away. Of course, there are always exceptions, like very tall and very skinny people or very short and very fat people. But we expect that most people will generally lies somewhere in the middle. So you get this kind of strong correlation from this. It’s clear that there is only one single factor that controls both of these observed variables. What is this factor? What we might think of it as size. So the single size factor controls both your height and weight. The physical manifestation of this might be a specific gene. Let’s suppose that the average person like you and me cannot directly observe genes. So for all intents and purposes, genes are hidden. That means there is a hidden variable called size, and this is causing the height and weight of an individual to be either smaller or larger. The key trick in principal components analysis is rotation and variance. Let’s suppose that we rotate the axis to align with the height and weight scatterplot. Equivalently, we can think of rotating the data points themselves. Now, what can we see when I want you to notice? Here is the key principle behind PCA. It’s that in one direction or one axis, there is a large variance for this picture. I’ve chosen the horizontal axis to be this axis with large variance in the other direction. There is a very small variance. It should be clear that this direction of large variance corresponds to the hidden factor we’re calling size. This direction is called the first principle component denoted by the unit vector Z one. The second principle component, which has less variance is denoted by Z two. You can imagine sort of sliding a point along these dots. At the far left end. We have a small size which corresponds to low weight and low height at the far right, and we have large size, which corresponds to high weight and high height. In other words, this single variable controls both height and weight. Now, in reality, you’re going to have hundreds or thousands of dimensions. So the previous picture you saw where we go from two dimensions to one dimension is just for intuition and practice. We might go from thousand dimensions to five dimensions or two dimensions. So the savings we get by transforming our data can actually be orders of magnitude. The best example for this in this class is obviously stock returns. So we have about 500 stock returns, but we know that they are not independent. They have correlation. We discovered this when we learned about portfolio theory. And as you know, when we have correlation, this is a good opportunity to use PCA, which again stands for principal components analysis. So here’s how we are going to approach these lectures in the rest of this lecture. I’m going to show you how to use PCA in code. We’ll also talk a little bit about what it’s actually doing and how to make use of the outputs. This will help you relate the code to the equation for the factory model that you saw earlier. Now, this lecture is not meant to be in-depth, but to just give you a very high level overview of what’s going on. This is what I’ll refer to as the beginner lecture in the next lecture, which I’ll refer to as the intermediate lecture. I’ll discuss what PCA is actually doing mathematically and how it can be applied. There is some very interesting applications of PCA, such as visualization, the noisy and decorrelation. But of course, to understand that you have to understand the math. Note that if you don’t have a strong background in math, then you may consider the following lectures optional and skip straight to the code. The following lecture will be the Advanced Lecture. The Advanced Lecture will prove to you that PCA actually does all the things that I’ve claimed it can do again. This one is optional. So if you are a beginner, don’t feel like you are required to watch in, these lectures will help you understand what we are doing in the code. But you can still do the code without that understanding. OK, so without further ado, let’s talk about the code, since we’re just going to use you learn it’s still three very simple lines of code, but it’s important to know what these lines are doing and what the inputs and outputs are. OK, so as usual, we’re going to start by instantiating a model. This time it’s an object of type PCA. The next step is to fit the model. Remember that since this is unsupervised learning, there’s no target that we call y while we pass into the fit function is X. So what is X? X should be a two dimensional array. Well, we have different dates along the rows and different stocks along the columns. Remember that the columns in machine learning are what we refer to as features and this is what we are trying to find the factors for. For example, Apple and Google may be highly correlated and controlled by a single factor. So that’s the kind of thing that we want our model to find. OK, so the next step is to call the transform function. So in PCA and other similar models, we don’t say that we’re making predictions, but rather doing transformations. This makes sense in the context of the picture that you saw earlier. As you recall, what we did was we rotated the data rotation is a kind of transformation. So it’s not prediction, per say, although it can be thought of like a prediction. Finally, note that off in the fit and the transform functions are conveniently made available as a single combine function simply called fit transform. So if you want, you can do both of the previous steps in just a single line. OK, so what’s this output Z. Well, if we think of the input X as an end by the matrix of features, we can think of Z as an end by K matrix of transformed features. Note that they have the same number of rows because they both must have the same number of samples. In our case, that would be the stock returns for a given date and the factors for a given date. However, D is different from K in our case, D is the number of original stocks and K is the number of factors. OK, so there are still more details that we don’t understand, one thing that’s probably pretty confusing is this. Earlier I said that the factory model is where we have the return, which we will now call X as a linear function of the factors which we will call Z. But our model seems to do the opposite. The model takes an X and gives us Z. That’s what we get when we call model that transform. So why does this appear to be backwards? Why are we going from X to Z instead of Z to X? In fact, to understand this, we have to understand rotation. So earlier in this lecture, I talked about rotating the height and weight variables so that they live on new axes, but how do we rotate feature vectors? Mathematically, I claim that the mechanism to rotate vectors is by doing matrix multiplication. So why does this make sense? Well, I encourage you to try it yourself, make up any vector and make up any matrix. Draw the vector on a piece of paper, then multiply the matrix by the vector. You should end up with a new vector, draw the new vector on the same piece of paper. You should observe that the new vector both rotates and stretches. The original vector that is the new vector is now pointing in a different direction and has a different length. Now it turns out that for IPCA, the matrices that we use will not stretch the original vectors and only rotate them. I’m going to say that without proving it. But if you want to learn more, you’re encouraged to inquire about my in-depth discussions on these topics. Just ask me about it on the Q&A, OK? And so basically what it boils down to is that if we have some feature vector of stock returns called X and we multiply it by a matrix called Q Transpose, we get the vector of factors called Z. Of course, when we call the fit function and Saikat learn, that’s how we actually determine what the Matrix Q should be. Now it also turns out that if we multiply the factor vector Z by Q, we get back to the original vector of stock returns called X.. So this is a two way transformation. When we want to go from A to Z. We multiply by Q transpose. When we want to go from Z to X, we multiply by Q. OK, so our goal right now is to get back to what we started with, that is a factor model. Well, we have a single stock return being a linear combination of factors. So far, we know that X is equal to Q time Z, but let’s see what happens if we write this out explicitly in terms of their components. Remember, I’m only interested in a single stock, so let’s say I’m interested in X1. I can see that X1 is equal to Q1 one time zwaan plus Q one two times two. Now let’s say I’m interested in X to X two is equal to a Q two one times one plus Q two, two times two. This is just by the rules of matrix multiplication. So what do we see. We see that X one, the stock return for asset one is just a linear combination of the factors one and Z two, it’s weighted by the coefficients Q one one and Kubuntu. We can make a similar statement about X two. OK, so hopefully now you are convinced that this implements the fact a model we discussed at the beginning of this lecture, it’s simply that by doing matrix and vector multiplication, we can calculate the returns for each stock simultaneously. That is, we get X one and two at the same time in one operation. So if you want to think of the factor model I described earlier in terms of betas, you can just imagine that a bunch of those are being calculated in parallel.

2. Statistical Factor Models (Intermediate)

In this lecture, we’re going to look more deeply into IPCA, we’re going to talk about what does PC actually accomplish? Why do we use it? What advantages does it have in this lecture? You’ll learn that PC actually does quite a lot more than what you might expect. OK, so to start this lecture, I’m going to give you a summary of everything that PC can do. And for the rest of his lecture, I’ll explain each topic in more detail. Again, let me remind you that this lecture is the intermediate level lecture. So if you don’t have the mathematical tools to understand this or you don’t care, then you can skip ahead to the code lecture. OK, so the first thing that PCA can accomplish is dimensionality reduction, that is we might start with 500 stocks, but when we transform them into factors, we end up with only three factors. We saw this in the original height and weight example as well. We started with two features, height and weight, and we discovered that they are only controlled by one factor, which is size. The second thing that PC can accomplish is visualization. Hopefully it’s not surprising that your computer screen is two dimensional and the physical world is three dimensional. So if you want to plot a five hundred dimensional object, humans don’t have the capability to comprehend what that even means. This is directly tied to dimensionality reduction as well, which was the first task. So if we reduce our dimensions from 500 to two, then we can plot these new dimensions and observe our data. Now, obviously, we lose some information in the process, but as you’ll learn later, we lose the minimum amount of information in some sense. The third test that PC can accomplish is Decorrelation, as you recall, stock returns are correlated, that is, there is redundant information contained in that PC removes this redundancy by having all the principal components be uncorrelated. So, for example, if one of the principal components refers to size, there won’t be another principal component that is also related to size. Redundancy is removed. Finally, the fourth task that PC can accomplish is the noisy, as you recall, from our height and weight example, the direction of large as the variants represented size. But there was another dimension with very small variance. Our interpretation of directions with very small variance is that it’s just noise. So by discarding those dimensions, we are removing noise from the data. OK, so now let’s discuss each of these topics in more detail. The first topic we’re going to discuss is dimensionality reduction. In fact, I’ve already told you pretty much everything you need to know. If we remove dimensions of the data, which are just noise, then equivalently we are just reducing the dimensionality. If, for example, we start with 500 dimensions and we go down to five, and what we’re saying is that those other four hundred ninety five dimensions are just noise. The practical question is how does this work? ENCODE, encode. As you know, we’re going to call it a fit transform function and we’re going to get back a matrix called Z. Previously, I told you that this matrix is of size and becae, which is only partly true, in fact, it’s actually owned by the same size as X. The original data, of course, this makes sense, given the fact that I told you earlier that PCA is doing a rotation rotation in and of itself does not remove dimensions, it just changes the directions. The key point is we choose K ourselves typically to be a value much less than be conveniently. When we get back the Z Matrix, it’s already organized so that the left most columns are the principal components with the greatest variation and the right most columns are the principal components with least variation. That is to say, they are sorted so we can pick the key most important principle components simply by ignoring the rest of the columns of Z on the right of this matrix. So that’s why we can think of the Z matrix as being and by K we’re taking the K leftmost columns of Z, and we know that we can do this because each column of Z is sorted by importance. Alternatively, there is an argument you can pass into the constructor of the class called end components. This is where you can specify explicitly, but as you’ll see, it’s usually a good idea to just do the full transformation and analyze the results yourself. Furthermore, even if you choose K smaller than D, the training part of the model still must be done in full. So you don’t gain much in the training process by choosing K smaller than the. You’ll see why this is in a later lecture. The next topic of this lecture is visualization. Now, this is pretty much self-explanatory after the previous topic, by reducing the number of dimensions, the two, you can make a two dimensional scatterplot of your data. Now, you might wonder whether you are losing information in the process of reducing your data dimensionality. The answer is yes. But the important question to ask is how much are you losing as per the previous slide? We already know that by choosing the two left most components of Z, we have chosen the two principal components with the most information for PCA. We can think of variants as analogous to information, although technically there are more rigorous measures for information. We know that PKA sorts each column of Z so that the dimensions with highest variance will come first. Well, it turns out that during the training process for PCA, which you will learn about next, we also get the variance for each of the dimensions and Z. So that tells us exactly how much each principal component contributes to the total variance. For example, we might find that the first principle component contributes 50 percent to the total variance, the next may contribute only 30 percent, the rest may contribute only one percent. So that’s the idea. And just as a side note, this is often another visualization we like to make when doing PCA. We don’t just want to visualize the data itself with the scatterplot. We want to visualize how faithful this representation is relative to the original data. And we can do that by examining how much variance is contained within each principal component. The next topic of this lecture is Decorrelation, it’s important to remember that uncorrelated this does not imply independence. So just because your variables are uncorrelated does not mean they are independent. However, being uncorrelated by itself can still be useful. Recall that correlation is just the scale to covariance. So when we say two factors are uncorrelated, that means their correlation is zero and this also means that their covariance is zero. Recall that the covariance is just the expected value of EXI minus Mooi times X minus Buji. This is sort of a generalization of variance. It tells us whether or not variable AI and variable J move the same as each other, move the opposite as each other, or are just not related at all. It turns out that the way we form Z, we make it so that each column of Z is uncorrelated. And this is important in terms of dimensionality reduction for dimensionality reduction. We don’t want any redundancy. If we have any redundancy, that means more dimensions can be removed. So by having each dimension be uncorrelated, we can interpret each factor as being a totally different hidden cause behind the stock’s return. The final topic of this lecture is the noisy, as you recall, one part of the cap is the noisy here. It says that any variation of a stock’s return beyond what is given by the market return is just random noise. But how do we distinguish between a signal and noise? We return to this idea of variance being of central importance. If variance is big enough, we usually consider that to be a signal. If the variance is small, we usually consider that to be noise. Intuitively, you can think of a radio which is trying to transmit a sinusoidal wave. The wave that it wants to transmit is a perfect sinusoid, of course, but the receiver will receive a fuzzy version of that sinusoid. That fuzziness is a result of adding random noise at small amplitudes so you can still see the original sine wave, but there is noise added to it. What pieces does is it separates the large variations from the tiny variations, and by removing the tiny variations, we effectively remove the noise. And again, in a practical sense, this is accomplished by simply ignoring the rightmost part of Z since Z is ordered in terms of decreasing variance. So instead of keeping the full and by D version of Z, we just take the leftmost part of Z of the shape, and by K work is much less than D. So in that sense, this is equivalent to dimensionality reduction.

3. Statistical Factor Models (Advanced)

In this lecture, which is the advanced lecture on PCEHR, we are going to prove that the PCA algorithm actually accomplishes what I said it would previously. Just as a recap, let’s remind ourselves of what PC can do. First, it does dimensionality reduction. We can reduce dimensionality by taking only the most important principle components. This is the same as the noise and visualization, because to do visualization, we need to reduce dimensionality to two to duty noises. We remove dimensions which are just considered noise. So what we actually want to show here is my claim that the Z matrix, which we get back from PCA, will actually be ordered in terms of decreasing variance. As you recall, we use variants as a proxy for information or importance. The second thing we want to prove is that the columns of Z, that is the principal components, are correlated from each other. In other words, their correlation or their covariance should be zero. OK, so our goal in this lecture is as follows. We’re going to derive PCA and I’m going to show you how the Q matrix we looked at earlier can be found. Furthermore, we’re going to show that by using this particular cue matrix, the Z that we get when we transform Exarchia will lead to the above properties. These properties, again, are that the principal components will be ordered by variance and they will be correlated. OK, so let’s begin. The trick behind PCA is that it’s nothing but Igen decomposition, in other words, finding eigenvalues and eigenvectors. So let’s remind ourselves about what that means. As we discussed earlier, we said that when you multiply a vector by a matrix, you can potentially do two things. First, you’ll rotate it so that it points in a different direction. Second, you’ll stretch it so that it becomes either longer or shorter. Eigenvalues and eigenvectors are special values in relation to a matrix specifically. Suppose we have a square matrix called a then if we have some scalar lambda and some vector V that satisfies the equation eight times V equals lambda times V, then we call this Lambda and eigenvalue and we call this vector V and I get vector. OK, so what’s special about the eigenvalue and eigenvector. Well what this equation is saying is that when we multiply V by the Matrix, this is the same as multiplying V by some constant lambda. But we know that when we multiply a vector by a constant scalar, it doesn’t change direction. Yes, if the scalar is negative, then it might reverse direction, but the direction is essentially the same. Thus the vector only changes in magnitude. So that’s what’s special about eigenvalues and eigenvectors. Eigenvectors, a special vector is that when you multiply them by a they are not rotated. Instead they are only scaled. How much are they scaled by the scale, by the value lambda. As a side note for this lecture and this course, eigenvalues will always be positive. So the eigenvector will not reverse direction. OK, so what metrics are we actually going to use, Igen Decompositional, the answer is the covariance matrix of X. Let’s think about what that means. Recall that you’ve seen the covariance matrix earlier in this course multiple times. So to recap, X is an N by the Matrix in our case. And is the number of dates or times steps in our data dataset d the number of stocks we can think of this as end samples of the D dimensional vector little X. The definition of covariance is the expected value of X minus such mean times. The same thing transposed as you know, we can calculate this practically by using the sample covariance. This just takes the thing inside the expected value for each sample little Zibi. Then we add them all together and divide by n also recall that we can do this calculation in matrix form. And again, remember that there’s some abuse of notation here because technically you cannot subtract a vector from a matrix, but we’ll assume it’s doing something like numpad broadcasting. So if you do this in code, it will actually work. OK, so what do we get after we do this calculation, we get a divided matrix that we call Sigma. Each element of Sigma tells us the covariance between each one of these stocks and each one of the other stocks. So Sigma, AIG is the covariance between Starky and stock. Note that this matrix is symmetric. So the covariance between Starky and Stock J is the same as the covariance between Stock J and Starkie. Furthermore, note that along the diagonal we have the variance. So the covariance between Stucchi and itself is just the variance of STARKIE. Now, let’s suppose we do Igen decomposition on this covariance matrix of X, what will this give us? Although the details are outside the scope of this course, we will get the unique non-negative eigenvalues and corresponding eigenvectors. For all practical purposes, we can assume that the eigenvectors are positive. There are several important properties of these eigenvectors that we need to discuss, the first property is that they all have length one. In fact, we can choose this to be the case. Recall that eigenvectors can be scaled arbitrarily to see this. Just consider the Igen equation. Iv. equals Lambda V.. If we multiply both sides by sea, we get a new vector V prime, which is just V scaled by sea. Obviously this also satisfies the Eigen equation so we can conclude that eigenvectors are not unique and therefore we can freely scale them so that their length is one. An equivalent way of stating this is that the norm squared is one or that V transpose multiplied by V is one. If you’re not convinced that these all mean the same thing, please write this down and worked it out as an exercise. The second property is that all the eigenvectors are orthogonal. So what does that mean? Geometrically, you can think of this as each eigenvector being 90 degrees or at right angles relative to all the other eigenvectors. In fact, you saw an example of this already when we looked at the height and weight graph. When we rotated this data, we rotated it in such a way that the new Axis Z one and C two, we’re still at right angles relative to each other. So what this means is that after we rotate the data in dimensional space, the axis of the new dimensions are still at right angles. Mathematically, what this means is that if you take two different eigenvectors and calculate the nanoproducts between them, you will get zero. I would recommend that if you’ve forgotten this, verify for yourself that if you have two vectors at right angles and you take the inner product, that you will always get zero. OK, so hopefully you’re convinced that this is true. And remember, this is only for eigenvectors that are different. If you take the same eigenvector and dot it with itself, then you will get one, which is what we discussed on the previous slide. If you want to know why this is true, here’s a short proof. Consider the quantity lamda times VSA by transpose times V subj. Let’s assume that I think are different. So VI and Vijay correspond to different eigenvectors. Well, we know that lamda sabai times VSA by is also equal to Sigma Times VSV I. By definition of the eigenvalues and eigenvectors I’ve grouped together Lambda invisible in brackets to make this more clear so we can replace Lambda Sabai with Sigma. So I have sigma times vsa by all transposed times v subj. Now, remember that when you take the transpose, the order of multiplication reverses. So this is the same as visa by transpose time sigma transpose types of subject. But we know that Sigma is symmetric. So Sigma Transpose is just sigma. Then we recognize that Sigma Times V Subject is also an opportunity to plug in an eigenvalue. Specifically, it’s just Lambda G Times V SJ. So we’ve discovered that Lambda Zibi times VSA by transpose times V subj is equal to Lambus of J Times Visa by Transpose Times Visagie. But how can this be? Remember that Lamda I and Lavada Jay are different, if they were the same, then they would have the same eigenvectors as well. So the only way that these two expressions can be equal is if the DOT product between VSA by and this object is zero and therefore we Sibai and we are orthogonal. All right, so all together, if the length of each eigenvector is one and each eigenvector is orthogonal with the others, then we can say that they are also normal for reasons which won’t be clear yet. What we would like to do is organize these eigenvectors into a matrix. We’ll call it Big V.. So the first column of Big V is V one, the first eigenvector, the second column of Big V is V to the second eigenvector and so on. And we just keep repeating this pattern until we get up to VMD altogether. This is called an ortho normal matrix. Sometimes people just call this an orthogonal matrix. However, I prefer ortho normal because it explicitly tells us that each vector is orthogonal and has unit length. What is interesting to consider is what happens if we multiply V by itself, specifically, what if we take a V transpose times V? Well, the first element at row one, column one is V one dotted with V one. As you know, that’s just one. The second element at row one column to is V one dotted with V two. As you know that zero. If you finish the rest of these calculations, you’ll recognize that the result is the identity matrix. You can do a similar calculation to show that V Times V transpose is also identity. So what does this imply? It implies that the Transpose V is also the inverse of this is because by definition, a matrix multiplied by its inverse is identity. OK, so how do we put this all together? Now, I realize this seems like a lot of completely random steps, but it will all make sense very shortly. Well, since we’ve put all the eigenvectors into a matrix, let’s put all the eigenvalues into another matrix. We’ll call this capital Lambda. It’s a diagonal matrix. Since all the diagonal elements are zero, the diagonal of this matrix contains the eigenvalues lambda one up to Lambda be. One important thing to remember is that each of the D eigenvalues is just a number. Typically we use software to solve for eigenvalues and eigenvectors like num pi. If you’re using Python in school, you usually learn how to find these four small matrices. But obviously you can’t do that if your matrix is 500 by 500. OK, so we have the eigenvalues and the eigenvectors that we just got back from Mumbai, we organize them into matrices. In fact, when you use Mumbai, what you get back is already in this form. But very importantly, we are going to sort these eigenvalues and the corresponding eigenvectors so that they are in descending order. Now, the reason why won’t be obvious now, but it will make sense in the end. So that’s one important fact. I want you to remember, although these eigenvalues are completely arbitrary numbers, we’re going to organize them in this big matrix lamda so that they are sorted in descending order, that is to say, from biggest to smallest. So LAMDA one is the biggest lamda two is the second biggest and so on. And then Lambda DX is the smallest. OK, so what’s next now that we have the eigenvalues and eigenvectors, assorted matrices, we can form the Eigen decomposition in matrix form. This is written as Sigma Times, Big V equal to Big V Times, Big Lambda. Now, this might look a bit strange since when we write this in the usual vector form V is on the right side both times. So as an exercise, what I want you to do is multiply out the matrix form, which will give you back these separate equations in vector form. You should be able to verify that these these separate equations are the usual vector equations for Igen decomposition. In other words, this ordering Stigmatise V equals V times lambda is the correct order. Now, there are two ways that we can rearrange this equation, which might be helpful. First, we can multiply the inverse on the right side. This will give us that sigma is equal to V Times, Lambda Times V inverse. We know that V inverse is equal to V transpose, so we can write that down as well. Now you see why this is called Igen decomposition. We have decomposed sigma into two parts. One part is a diagonal matrix of eigenvalues. The other part is an orthogonal matrix of eigenvectors. Another thing we can do is multiply the inverse on the left side. This gives us that lambda is equal to of inverse times, Sigma Times V and again, we replace the inverse with V transpose. So what does this say? Usually when we do this, we say that we’ve diagonals sigma. That is, we found some equivalent representation of Sigma in terms of a diagonal matrix lambda. Now again, these might seem like arbitrary curiosities, but they will come together soon. The next step is to remember what we are doing, we are doing PCA, where our job is to find some matrix cue so that we can transform any vector X into a vector of factors called Z. We can also do a reverse transformation by using cue again. Well, what is this cue matrix? This cue matrix is just v the matrix of eigenvectors that we just found. Now that we’ve done all the groundwork, the rest of this lecture will be easy. All we have to do now is prove that using this choice of cue will lead to the properties of Z that we discussed earlier. Just as a reminder, this is that each column of Z, which is a factor, is ordered so that the most important factors are on the left and the least important factors on the right. Furthermore, each column of Z is uncorrelated with the other is that is to say the factors are not redundant. OK, so next, just this a sort of arbitrary task, let’s try to calculate the covariance of Z. If we use V as our transformation matrix, we can start with a definition of covariance will denote the covariance of Z by Sigma Z. It’s the expected value of Z minus the mean of Z multiplied by his transpose. But we recall that Z is just V transpose times X so we can replace Z with V transpose times X. We can also replace the mean Z with V transpose times the mean of X. The next step is to factor out the V matrix, since it’s not a random variable on the left side we have V transpose and on the right side we have V. But the important part is what we have in the middle. As you can see, it’s just the covariance of X so we can replace that with Sigma X.. Well, you might recognize this as something we’d arrived earlier. In fact, this is just lambda the matrix of eigenvalues. So what does this say? It’s saying that the covariance of Z is just the matrix of eigenvalues where those eigenvalues were derived from X. What does that mean? It means that any of the diagonal values of Sigma Z are zero because by definition, Lambda is a diagonal matrix. What does that mean? It means that the covariance and equivalently the correlation between any factor and Z and any other factor in Z is zero. That is to say, the factors in Z are uncorrelated, as you recall. This is one of our claims from earlier. Furthermore, remember that we sort of arbitrarily decided to sort the eigenvalues in descending order. So the eigenvalues are sorted from biggest to smallest. But what do these eigenvalues represent? Well, these are the variances of each factor in Z. So Lambda one is really the variance of factor one. Lambda two is really the variance of factor two. Thus each factor in Z is also sorted in order of descending variance. The variance of factor one is the largest. The variance, a factor of two is the second largest and so forth. And remember that we are using variants as a proxy for information, so the first principle component has the most information. The second principle component has the second most information and so on. OK, so that concludes our derivation of principal components analysis. In this lecture, we proved that PCA has the properties we described in the previous lectures and therefore is suitable for the task that we claim that PCA can do. As you recall, these properties were there when we transform the data. The new data is uncorrelated. Furthermore, the columns are ordered in terms of importance where importance is measured by variance. This allows us to perform at dimensionality reduction, viz. the noisy and decorrelation. And this allows us to build a statistical factor model such that we can choose the most important factors that affect US stocks return and ensure that each of these factors are not redundant.

4. Statistical Factor Models (Code)

In this lecture, we are going to apply PCA in code to stock returns. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so there are two data sets in this lecture that we need. First, we’re going to want the close prices for all the stocks in the S&P. This is going to be the data that we perform on. Next, we’ll need the S&P itself. You’ll see how we make use of this at the end of this lecture. Next, we’re going to import a number pendas, a matplotlib, as usual. Next, we call GSV to load in the closed prices data set. Next, we call DFG head to remind ourselves of the format of this data set. Each column is a different stock, and Ejiro is the date, each element is the stock price for a specific date for a specific stock. Next, we call it the shape to check how many rows and columns of data we have. So it looks like we have over two thousand rows, which means over two thousand days and 480 columns of data, as a side note, it’s useful to remember one important property of matrices, which is the rank, the rank of a matrix, tells you how many linearly independent rows or columns it has as a consequence of this, when we calculate the eigenvalues of the covariance, that’s how many positive eigenvalues we will have. So to give you an example of how this might affect us, suppose that we only looked at one year of data. That’s approximately two hundred fifty two rows. So we would have two hundred fifty two rows and four hundred eighty columns. In this case, the rank of our data matrix would only be two hundred and fifty to the minimum of the number of rows and columns. So when we go to calculate the covariance matrix, there will only be two hundred fifty two positive eigenvalues and the rest will be zero. This is why we would typically like to have more rows than columns in our data sets. OK, so we know that there are some missing values in our data. One easy way to get rid of problematic rows is to just drop any column with more than 10 missing values. If we fill these in, we’re going to get lots of zeros, which might affect our results in order to do this. I’m going to call the drop in a function we pass in access equals one to say that we want to drop columns we pass in threshold equal to the length of the data frame, minus ten to mean that we want to drop any column with 10 or more missing values. Note that this is not obvious from the documentation, so you have to verify that this does what you want it to do. We say in place equals true so that we don’t have to make a new copy of our data frame. OK, so let’s check the shape of our data frame again. We see that it now has four hundred thirty columns, so 50 columns of our data set had 10 or more missing values. Next, let’s call the dot head to see what’s in our data frame. Next, let’s call is N.A to check how many missing values are still in our data frame. As you can see, there are about 3000 missing values still remaining. Next, we call the fill in a function using forward filling. Next, let’s check how many missing values are left. OK, so now there’s just one missing value left, which isn’t too problematic. Next, we call the fill in a function again using backwood filling. OK, so let’s check in again and see how many missing values are left. So there are now zero missing values left as expected. Next, we’re going to build a new data frame which will contain the returns for each stock. As you recall, what we have currently are prices. But will we want to correlate are the returns? We’ll start by grabbing all the data from our data set. To do this, we take the index of the data frame called a unique function and call the sword function. This is probably not necessary, but it’s called I Copied from earlier. Next, we’re going to create a new data frame with only the index using the data we just obtained. We start from the index one, since we know that when we calculate the returns, the first value will be missing. Next, we live through each stock ticker in DFG columns inside the loop, we calculate the log return by taking the log of the close price and then taking the difference. Next, we select the returns from index one up to the end and assign this to our data frame called returns. Next, we do it head to ensure that our data frame looks as expected. Next, we convert our data frame to an umpire called X. The next step is to standardize our data, that is we want each column to have mean zero and variance one. Let’s think about why we want to do this. Remember that the job of PCA is to find the direction of greatest variance. This means that it is not invariant to variance. So why might this be important? Let’s suppose that we’re doing machine learning and that we have two features. One is very important and the other is not that important. Let’s say that the really important feature is the crime rate of a neighborhood which is on a scale from zero to one. Let’s say that the unimportant feature is house price, which is on the scale of hundreds of thousands. In this case, house price completely overshadows crime rate because its scale is larger. However, as we noted, this feature is unimportant. Yet PCA will focus on this feature because it has the greatest variance simply due to having the greatest scale, which is not what we want. What we would actually like to do is to put each feature on equal footing so that PCA will not be biased towards any particular feature and we can accomplish this by making every feature have univariate. OK, so to do this we import standard scalar and then we instantiate an object of type, standard scalar. Next we call fit transform and we overwrite X with the new transform data. Next, we’re going to run a picture. So, as usual, we import IPCA and then we instantiate an object of type. Next we call fit transform on X and we get back Z and encourage you to check the shape of Z to confirm that it has the same shape as X.. They should both be and by the. The next step is to plot the variance of each of the principal components. We do this by calling model, explain the variance ratio. This is a ratio because it normalizes the variance so that they all come to one. That is each variance you see in the plot will be a percentage. OK, so what do we see? Well, we see that a majority of the variance is explained by the first principle component. It looks to be about 35 to 40 percent after this. The variance seems to drop off quite fast. In order to get a closer look, we’re going to plot just the first 10 values of the explain the variance ratio. We can see from this plot that the first principle component explains over 35 percent of the data, while the second principle component explains about five percent. After this, the variance drops even further to about two percent or one percent. Another common plot is the cumulative variance ratio in this plot. We calculate the cumulative some of the explain the variance ratio. This is accomplished by calling the function consomme. The idea is to ask if we keep one principle component, how much of the total variance. Have we accounted for? If we keep two principal components, how much of the total variance have we accounted for? So for each number of principal components that you keep, you get a sense of how much variance is explained relative to the total. OK, so what do we see? Well, we see that at the beginning there was a steep rise due to the first principle component explaining nearly 40 percent of the data. After that, we have diminishing returns for each principal component that we include. This actually lends some credence to the cap. And the cap says that there is one very influential factor which we refer to as the market. After that, everything is pretty much just noise. This noise seems to be individual to every individual stock, which seems to be what we’re seeing. If this were not the case, then there would be other principal components which explain a larger percentage of the variance. Instead, we only have one very significant principal component and the others are very small in comparison. OK, so just as an experiment, what I would like to do is plot the first principle component of the transform data and compare that to the S&P. Remember that the first principle component is the most influential hidden factor. So to do that, I’m going to call people to apply and I’m going to pass in the first column of Z.. OK, so that’s how that looks. Next, I’m going to create a new data frame to store this principal component, will use this new data frame to plot this against the S&P 500 with corresponding dates. So first, we instantiate a new data frame using the same index as the returns data frame. We then create a new column called PC One and a sign that the first column of the. Next, we load in the S&P 500 using pedigreed CSFI. Next, we calculate log returns for the S&P 500 by taking the log and then taking the difference. I also called Drop N.A in case there are any missing values. Notice something strange here, which is that I negate the log returns. So why do I want to do this? Well, let’s remember that eigenvectors, which are what we use to transform the data with PCEHR, are not unique. If V is an eigenvector, then so is minus V. There’s still meets the requirements for IPCA since V has length one, minus V also has length one. So there’s no guarantee that the principal component will have the sign that we require. In this case, I noticed that it was reversed relative to the actual SNP returns, so I’ve negated the SNP returns so that they line up. And if we scroll back to our earlier plight, you can see that both of these plots do look very similar. The next step is to create our joint data frame, so we’re going to take ZDF and we’re going to call the joint function passing in the spy returns. Next, we’re going to call joined Dot Head to see if our data frame has the right format. OK, and it looks like it does. There is one small issue, though, which is that if you recall, these values are not on the same scale. The first principle component is the rotation of the standardized stock returns. SBI is currently the actual stock returns of the S&P 500. So in the next block of code, we’re going to fix this issue. We’ll start by renaming the columns to PC one and S.P.I, which make a little more sense. Next will create a new standard scalar object called scalar too. Next, we call it transform on the joint data frame. We assign this back to the joint data frame under the columns PC one and SPI. OK, so the last step will be to make our plot. All we have to do is call joins the plot with a few options to help us see our plot better. First, I’m going to fix eyes to 10 by five. Optionally, you can use an interactive plot or plot this on your machine if you would like to zoom in. We also set Alpha to zero point seven so that the lines have transparency. This will be useful since they are going to overlap. OK, so let’s run this. So we see that, as our visual inspection suggested earlier, our first principle component very closely matches the S&P. So why does this make sense? This makes sense in terms of the cap in the cap says that there is one major factor that influences all stock prices, and that’s the market. When we performed principal components analysis, we found this one factor. That’s why we had one principal component with a disproportionately large amount of variance. Furthermore, what is the market? Well, we know that a good proxy for the market is the S&P 500, and thus it’s not surprising that the first principal component very closely matches the S&P 500.

9. VIP Regime Detection and Sequence Modeling with Hidden Markov Models

1. Why Sequence Models (pt 1)

In this lecture, we are going to motivate the use of sequence models for modeling stock returns. So let’s start by recalling one of the themes of this course. This, of course, is not about making perfect predictions. And in fact, a lot of the course is about why you cannot make perfect predictions. It’s about understanding why predictions like these, which usually seems are not useful at all. And not only that, they are worse than simpler models like Anand’s and Arima. Most of this course was getting you to recognize what you cannot do. So the question remains, what can you do? And I hope you’ll recognize that this section partially answers this question. OK, so earlier in the course, you’ll recall that we modeled stock returns, we noted that although initially they look like they might be normally distributed, they are not actually normally distributed. Instead, they have fat tails and skinny shoulders. Two ways we modeled this kind of distribution was with the T distribution and with Gaussian mixtures. We recognize that when we drew samples from two Gaussian distributions with different variances instead of just one, we got the fat tails we were looking for. But we also recognize that stock returns are not quite stationary. Stock returns exhibit what is called volatility clustering. That is to say, when we see high variance, it is usually surrounded by high variance. When we see low variance, it is usually surrounded by low variance. This clearly violates the definition of stationary, although the statistical tests we use for measuring stationary, we’re not able to detect this. So this section on sequence modeling is all about answering the question, can we model this time dependent on volatility? In this section, our model of choice will be the hidden Markov model or the HMO. We looked at Markov models earlier in this course when we discussed the random walk hypothesis. This is similar to that, except now our Markov model is over a set of discrete states. But before we even discuss Markov models or hidden Markov models, I want to discuss graphical models. This will help us understand the structure of our model and how we can build it up from simple to complex. So earlier in the course, we looked at the Gaussian mixture model, let’s review that again. But in the context of a graphical model, in the graphical model, it looks like this we have some hidden cause or latent variable called Z. This is represented by an unshattered node. Unassociated means that we do not observe this value. Then we draw an arrow going from Z to X. X represents the stock return or the observation that we actually do see. Since we do observe this, it is shaded in note that some books or resources may reverse this convention or they might simply use different colors and not shades either way. The important thing is that you remain consistent in your own work, but don’t freak out if you find out that different resources use different conventions. OK, so let’s think about the physical interpretation of this graphical model. I like to think of it as God flipping a coin. If the coin lands on heads, then we sample the return from one Gaussian, if the coin lands on tails and we sample the return from a different Gaussian. Of course, God can’t tell us the result of the coin toss directly, so all we get to see is the sample from the Gaussian in our case. That’s the stock return. OK, so that’s the physical interpretation. God flipped a coin and then the clouds decide how much it’s going to rain based on whether the result was heads or tails. If the result was heads, the clouds get very excited and rain a lot. And if the result is tails, the clouds are very relaxed and decide not to rain that much, if at all. We can only measure the amount of rainfall, but we cannot measure the result of God’s coin flip. It also helps to think of other kinds of graphical models to build your intuition. Here’s a very simple one representing your own coin toss in this scenario. All we are concerned with is the results of the coin toss, whether that’s heads or tails. Since you can observe the results of your own coin toss, this note is shaded in. Furthermore, there is no process involved with different random variables. The only random variable is the coin toss result itself. Therefore, our graphical model is just a single node without any edges. Here’s another graphical model known as Naive Bais in this graphical model, everything is observed. To give you an example, suppose we are trying to predict whether or not an email is spam. So that’s what WI represents in supervised machine learning. We call this the label in our model. This Y has a direct influence on certain attributes of the email itself. Let’s suppose that in this simple case, we are looking at two attributes or two features feature. Number one is whether or not the email is from someone in your contact list. We’ll call that X1 feature. Number two is whether or not that email contains the phrase make money. We’ll call that X two so you can imagine why the label has an effect on these probabilities. If Y is spam, then we expect the probability of the email being from someone in your contacts list to be lower. If Y is spam, then we expect the probability of the phrase to make money to be higher, OK, and we can measure these probabilities using simple counting as usual. The important part of this picture is that you understand how each of the nodes affects the others and the physical interpretation, the direction of influence is the direction of the arrows. Here’s another interesting fact, if we compare the bais classifier with the Gaussian mixture model, we can see that they essentially have the same structure. Note that in this case, this is a non naïf based classifier, whereas on the previous slide we were looking at naïf base. In this case, the entire observation is wrapped up in a single vector X and it’s not possible to separate it into X one, X two and so forth. In any case, don’t worry so much about these details, although if you want to learn more, I’ll be happy to provide you with references on the Q&A. What is important to recognize is that these have the same structure in the base classifier. We have a coin flip called a Y, which we are allowed to observe and then this produces an observation X which is dependent on Y in the GM. We have a coin flip called Z, which we do not observe. And then this produces an observation X which is dependent on Z. In both cases, our observation distribution was dependent on the result of a coin flip. The only difference is that in one case we were not allowed to observe the result of the coin flip. Finally, note that the arrows represent conditioning. So if a node does not have any arrows going into it, it’s represented by a marginal distribution that would be either P of Y or Z. Then since we have an arrow going to X, the next distribution would be PVCs given Y or Z. This makes it explicit that the distribution of X is a function of Y or Z. OK, now that you understand the basics of graphical models, let’s look at the Markov model. The Markov model is where we have a set of states and the probability of moving from one state to the next depends only on the state that you came from, but not any state before that. To give you a realistic example of this, suppose you are modeling your state of sickness and health. So you have one state which represents being healthy. You have another state which represents sneezing when you are sneezing. It could be the case that you are about to catch cold or your immune system may be able to fight it off and you could go back to healthy. Finally, you have another state which represents having a cold. Now we also have associated probabilities for each of these arrows. For instance, the probability of being healthy and staying healthy is zero point nine nine. That is to say, if you’re healthy one day, it’s very likely you will remain healthy the next day. Of course, there’s a small chance that the next day you may develop some sneezing, which might signal an oncoming cold. Perhaps the probability of going from healthy to sneezing is zero point zero zero nine. Finally, there is a small chance that the next day you may develop a full blown cold. The probability of that is zero point zero zero one. Note that these three probabilities sum to one. Now let’s turn our attention to the sneezing stay in this day. Let’s say that you have a pretty even chance of just going back to healthy the next day or becoming sick the next day. So the probability of going to either stay at zero point five, note that the probability of going back to itself is zero since these probabilities must come to one. Finally, let’s look at the state where you have a cold. In this state, you have a pretty high chance of still having the cold the next day at 80 percent. You also have a decent chance of getting better at 20 percent. So why does this make sense? This is because when you have a cold, there is a pretty good chance it will last more than one day. Therefore, the probability of staying in the state on the next day should be greater than 50 percent. However, it should not be as high as going from healthy to healthy because typically we spend much more time being healthy compared to having a cold. So hopefully these probabilities make sense to you and hopefully this helps you understand what a Markov model is, the really important thing to notice about this is that whenever we decide which state to go to next, the probability depends only on what state we are in. For example, if we are healthy today, we know that the probabilities for tomorrow are zero point nine nine zero point zero zero nine and zero point zero zero one. This is the case regardless of where we were yesterday or the day before that. That’s why this is a Markov model. Now, I just realized that this diagram may be terribly confusing, not because it’s confusing by itself, but because it is not represented as a graphical model like the diagrams we were looking at earlier in this lecture. Note that for those graphical models, each node represented a different than a variable and the arrows represented the conditioning of those random variables. In this current diagram, all the nodes represent the same rent, a variable that is your state of health. And each arrow represents the probability of going from one value of that state to a different value. OK, so how would a Markov model look in graphical model form in this case, your state of health on each day or each time step is a separate random variable so we can call the state of health on D’Haiti X of T, so we have X of one of two, X of three and so forth. Notice a few important things. Firstly, each X of T depends only on the previous day, X of T minus one. That’s what makes this Markov X of T does not depend on any day before X of T minus one, nor on any day in the future. Otherwise we would have to draw arrows connecting those days. Furthermore, notice that each node in the graphical model encapsulates the entire distribution and set of states so X of T can be healthy sneezing or having a cold, but we don’t actually see those on this diagram. Furthermore, we also cannot see the type of distribution effects of T could be a discrete random variable with one thousand states or it could be a continuous random variable. But either way, this diagram does not tell you this information. Finally, note that unlike the previous diagram, these arrows do not represent the probability of transitioning from one state value to the next. These arrows represent dependencies just like the graphical models for games and naïve bears that we looked at earlier. As another side note, recognize the fact that what we described previously is a discrete state Markov model. That’s why we can draw it as a diagram with nodes and arrows. What we drew was a state transition diagram showing each of the individual discrete states and the probabilities of transitioning. This is opposed to the random walk model we discussed earlier in the course. That was also Markov, but that was a Markov chain over continuous states of which there are an infinite number of values. Therefore, the Mark property is still true, which means that each state depends only on the previous day. However, it’s not possible to draw it in the same way since our state space is continuous. On the other hand, it can still be represented by a graphic model where each node it can represent any kind of distribution and each arrow represents a statistical dependency.

2. Why Sequence Models (pt 2)

OK, so the next thing we are going to do in this lecture is to bring all these concepts together to form the hidden Markov model. We’ll start with a diagram which is in non graphical model form, since I think this might be a bit more intuitive. However, I think they both provide insight into how the model works. It’s just insight from a different perspective to remind you in this non graphical model form, this means that each state will be explicitly shown and each edge is the probability of going from one state to another. So let’s say we have three hidden states. They are healthy, sick and allergies. Sick means you have a cold and allergies means you are suffering from seasonal allergies. Let’s suppose these are hidden, which makes sense, since sometimes you might have a runny nose, but you can’t figure out the exact cause. Maybe it’s allergies, or maybe it’s because you have a cold or maybe for some other unknown reason, but you are otherwise healthy. Now, just to make this interesting, let’s make the runny nose a continuous variable to be some measure of how runny your nose is. For example, it could be the volume of mucus that you ejects from your nose on that day. We can model that with a Gaussian distribution. OK, so first, let’s consider the parameters of these Gaussian distributions. If we look at the healthy case, we can see that it’s a Gaussian with Mintern and Variance 10. As a side note, I’m just making these numbers up, so don’t try to look them up in a science journal. They only make sense relative to each other. In the SEC case, we can see that it’s a Yasiin with means 60 and variance 50. This makes sense because when you’re sick, you should expect to have a runny nose, which means more mucus. Finally, in the allergies case, we have a Gaussian with meaning 80, variance 50. This means that on average, allergies give you more of a runny nose than having a cold. Also, notice that I’m sticking to the convention, that the observations are represented with the variable X and the head and states are represented with the variable Z. OK, so now let’s look at the head and states for the hidden states, we can see the probability that we go from one hit in state to another. If we are healthy, then there is a 99 percent chance that we will be healthy tomorrow. There is also a zero point nine percent chance that we will have allergies tomorrow and a zero point one percent chance that we will have a cold tomorrow. On the other hand, if we are in the allergy state, we have a 97 percent chance of still having allergies tomorrow while we have a two point nine percent chance of going back to healthy and a zero point one percent chance of catching a cold. This makes sense because when you have seasonal allergies, it lasts for the whole season. That is to say, if you have allergies today, then it’s pretty likely that tomorrow is still part of allergy season. Finally, if we have a cold, then we have an 80 percent chance of still having a cold tomorrow, a 19 percent chance of being healthy tomorrow, and a one percent chance of having allergies tomorrow. One feature of this diagram that I want you to notice is that similar to the plain Markov model, we can see that every node depends only on one other note. That is to say, the amount of runny nose you have, meaning each of the three Gaussians depends only on the state that you’re in today. In other words, if you are healthy today, then we must sample the runny nose amounts from the healthy. Yasiin, if we have allergies today, then we must sample the runny nose amount from the allergies Gaussian. OK, so going back to graphical models, here’s another way to look at the HMO in this view, we can see the time dependencies explicitly notice how each head and state now has a time index and each observation X also has a time index. Also, notice that now each state variable encapsulates all the values of that state. So we are not explicitly showing healthy, sick analogies. Instead, this is implicit in one symbol zaev t. Next, notice how each of these variables are related. This will help you understand why the model is Mark-Up. We can see that each Z depends only on one variable, the past Z. That is easy to depends only on what Z three depends only on Z two and so forth. So whether I am healthy today depends only on whether I was healthy yesterday, but not on two days before. Three days before. Furthermore, notice how the relationship between the observations and the head and states is also Markov exact time t depends only on Zayat time t, that is to say, my amounts of runny nose does not depend on whether I was healthy or sick yesterday or tomorrow. Of course, logically that makes sense. The amounts of runny nose I have today is dependent on my state of health today. Furthermore, notice that X of T does not depend on any other X’s, you might assume that the amounts of runny nose I have today might depend on the amount of runny nose I had yesterday. But this graphical model shows us that under the modeling assumptions, this is not the case. OK, so why did we discuss all of this and what does this have to do with modeling stock returns? Well, now we can use the same graphical model, but just replace the variables now instead of the hidden variable being the state of health now, it will represent some kind of state of the economy, which we will call the financial regime regime as a financial term. That essentially means state. We can think of it as the state of the market or a state of the economy or something like that. I don’t want to be too formal in this cause. Typically, we can talk about the high volatility regime and the low volatility regime. There are other characteristic behaviors inside these regimes, too, but the amount of volatility is the most fundamental. OK, so the Hennan state now represents the regime and there are two values for the state, low and high volatility. Then the observations that we make are the returns are low returns on an asset. Now we can also use a Gaussian distribution to represent these returns. So why does using a Gaussian make sense now when previously we saw that it failed to model stock returns accurately? Going back to a non graphical model, we can see why this makes sense when we are in a low volatility regime. The return is sampled from a Gaussian with low variance when we are in a high volatility regime, the return of samples from a Gaussian with high variance. So why are we now allowed to use a Gaussian to represent the return of distribution? Remember that the return to distribution only looks fat tailed because we ignore at a time that is to say, we looked at the histogram of all returns, assuming they came from the same single distribution. But then we realized that if we use the Gaussian mixture, we could more accurately model the return distribution by assuming that some returns came from one Gaussian and other returns came from the second Gaussian. That means on some days the returns were sampled from one gassier and on other days the returns were sampled from the other Gaussian. However, there was still no time dependent’s in that model, we were just randomly flipping coins to decide which Gaussian to sample from with the hidden Markov model, what we are saying now is that we are still sampling from two different Gaussians, but there is structure and how we choose these Gaussians. So let’s go back to our graphical model previously when we looked at the games, the Z variable was an independent, biased coin toss. That is, we were flipping a biased coin to choose either the fat Yasiin or the skinny Gaussian. But each time we flipped that coin, the result was not depending on past coin flips. Now, with the map, what we are saying is that we are still flipping this bias coin, but the coin flips are not independent. Instead, the result of each coin flip depends on which regime I was in previously, and so hopefully that makes sense in terms of volatility clustering. We know that if we are in a regime with high volatility, then there is a high probability that we will stay there if we are in a regime with low volatility. Then there is a high probability that we will stay there. In other words, those coin flips are not independent like in a Gmod, but they are more like an HMO where each coin flip depends on the last state. So I just realized that the title of this lecture is Why Sequence Models, but as of right now, we haven’t actually answered this question explicitly. But I hope that given what we just discussed, the answer is clear. We want to model the sequential nature of regimes switching because we know that volatility clustering exists. The GM was able to model stock returns when we assumed stationary, but we know that a non stationary model is more accurate. Furthermore, because a mixture of Gaussians leads to a fat tail distribution, if we were to look at the stock returns sampled from an HMO after throwing away the time dependence, we would again get back a fat tail. Giammo. In other words, the HMO is just a more general model. It not only models the sequential dependencies of stock returns, but it’s still models the fat tail ness of the GML. To understand this more clearly, we can use what is called plate notation in plate notation, we can express repeated samples, that is for the GMAR. We explicitly state the time dependence. So instead of just Z, we say Z of T, and instead of just X, we say activity in the corner. We say that little T goes from one up to big T.. Now we can compare the GMs and they h m m more directly since now both graphical models have a time index. So here’s the difference. It says that for the GM, each side of T is a coin toss, but it’s a coin toss that just comes out of nowhere. It does not depend on anything. On the other hand, for the HMO, each side of T is still a coin toss, but it’s a coin toss that depends on the results of the last coin toss. So that’s the big difference between GMs and achievements with GMs are coin tosses are independent with H.M.S. are coin tosses are dependent on the last coin toss and they form a Markov chain. However, notice that the observation X of T is always dependent on only CFT in both cases and there is no dependence on any other random variables. To help you understand this further, here’s yet another way we can look at this model now instead of using the more compact plate notation I’ve expressed to each of the Zaev T’s explicitly showing the one as a two, three and so on. So now you can see the only difference between the amendment, the GM, is that there are no arrows connecting the one with Z to Z, two with the three and so on.

3. HMM Parameters

In this lecture, we are going to look at the A.M. a little more in depth and discuss the parameters of an HMO. As you know, most machine learning models have parameters which are found using the learning process or the training process. An example of this is neural networks where you want to find the neural network weights. Another example is the simple Gaussian, where you want to find the mean and variance in our HMO. The Gaussian is actually part of the model. So we’ll have means and variances as well. But the more important part for us is going to be to focus on the state transitions. Let’s go back to our Markov model diagram, as you know, in this form, each arrow on the diagram represents a transition and each number along the arrows is the probability of making that transition. To bring this closer to our specific example. Let’s consider our states. We’ll be using two states which represent low and high volatility. Note that in general, the number of states can be considered a hyper parameter that you can optimize using the usual methods such as cross-validation. This is because you don’t actually know what the hidden states are, how many of them there are, and so on. However, this is a sort of unique scenario where we do have some idea of how many states there are and what they represent. So we can use that knowledge to constrain the degrees of freedom of our HMO in particular by setting the number of states to be, too. So here’s something we might expect to find. If we look at a typical stock return sequence, we can see that there are generally long periods of low volatility and shorter periods of high volatility. This should remind you of our healthy and sick example. So a reasonable guess is that the probability of going from low to low is, say, 95 percent, and that means the probability of going from low to high is five percent. On the other hand, maybe the probability of going from high to high is 80 percent and the probability of going from high to low is 20 percent. This makes sense because from these pictures, it’s evident that we spend more time being in the low state. This is just like our healthy and sick example where we spend more time being in the healthy state. Note that in general, these probabilities are called state transition probabilities without considering any specific examples, we can see that our states are represented by the indices one up to em. Then we can say that the probability of going from state to state J is P of time T plus one equals J given Zayat time T equals I. OK, so that’s just four. When we want to write down these probabilities mathematically, it should be clear that I and J both count from one up to M. Furthermore, note that these probabilities do not depend on T. That is the transition. Probabilities are the same today as they were yesterday. They will be the same tomorrow and so on. It’s worth considering how many of these probabilities we will have. Well, since I has impossible values and Jay also has impossible values, then it should be clear that in total we will have more times. M a probability is. Since we have M squared probabilities and they are constant over time, it’s convenient to store them in a matrix. We call this matrix the state transition matrix and we typically denote it by the letter A.. So AIG is the probability of going from state to state J. Why is this important? It’s important because this is what we will see in the code. Furthermore, let’s recall that we also have the observation probabilities from each state. So for the low state, we will have one Gaussian, which is expected to have small variance for the high state. We will have another Gaussian, which is expected to have large variance as before. We recall that by sampling from two Gaussians with a different variance, the overall effect will be that we have a fat tail distribution and this is what we want since stock returns come from fat tailed distributions. Now, one question you might have is, how do we actually find this matrix along with the Gaussian means and variances given only a time series of stock returns in this lecture? I want to give you the basic intuition for this problem, but in general, the answer is more complex. So let’s start by assuming that we know which regions are high and which regions are low. That is to say, for every day in my data set of stock returns, I’ve labeled them all high or low. Then finding these probabilities is easy. All we have to do is count. For example, the probability of going from high to high is the number of times I’ve transitioned from high to high, divided by the total number of high days. Conversely, the probability of going from high to low is the number of times I’ve transitioned from high to low, divided by the number of high days. As an exercise, you might want to think about what expressions we would use to measure the probability of going from low to high and low to low. And note that although I’ve said we are assuming we know which days are high in which these are low, this is not unrealistic. This is one of those scenarios where we can use domain expertise to help with our data. If we know, for example, that there was some big event that caused a lot of people to buy or sell their stocks, and we know that this was a period of high volatility. However, there is a more general way to train incomes that do not involve having to label the data yourself. We’ll discuss that more in the coming lecture’s. So as a bonus for this lecture, I want to give you a very intuitive idea of how we would train NAMM given a time series sequence. This is a labeled time series sequence where each day is marked high or low volatility. Note that in general, this is not the case. OK, so suppose that this is our data set with 10 days of data. Note that each day has been labeled with an L to mean the state is low ends in H to mean the state is high. So we start with three low days, followed by three days, followed by four more days. OK, so let’s remind ourselves what we need to find. We need to find the probability of transitioning from every state to every other state. That is the state transition matrix. In addition, for each of these states, we need to find the mean and variance of a Gaussian. I would recommend that you sit down and take out a piece of paper and write these out for yourself. OK, so let’s start with the probability of going from low to low, we transition from low to low five times, whereas we are in the low state seven times. However, note that we will not count the final state because this actually terminates the sequence in practice. You might have a dummy state for the start and end. So let’s make the denominator six. Therefore, P of low given low is equal to five over six. We can also see that we transition from low to high one time. So high, given low is equal to one out of six, we can do the same thing for when we start from the high state. The number of times we transition from high to high is two and we are in the high state three times. Therefore P of high given high Ziegel to two out of three. Consequently, of low given high is equal to one out of three. OK, so this gives us our state transition matrix. We have five out of six, one out of six, one out of three. Intuitive three. OK, so now let’s calculate the mean and variance of the Gaussians for each state, for the low state, we have the returns zero point one minus zero point zero five zero point zero two minus zero point zero one minus zero point zero eight zero point zero six and minus zero point zero two. For the high state, we have the returns zero point three minus zero point five and zero point for what we want to do is calculate the sample mean and sample variance for each of these sets of numbers. After doing so, you should get the values that you see on this slide. Again, it’s best if you try this yourself, either on paper or using a computer. OK, so I hope that this idea of training in Yemen, although it sounds complex, is actually just a matter of calculating probabilities.

4. HMM Tasks and the Viterbi Algorithm

In this lecture, we are going to continue our discussion on natuman parameters and how we can use in each of them specifically, we’ll be focused on the three main tasks that each of them can do and will briefly go over the intuition for how each of these works. In addition, we’ll discuss why these tasks are important. So you understand their purpose in real world application. Note that this lecture is intended only to be a brief explanation of these tasks, not a complete derivation or in-depth discussion. This is just like the other models we discussed in this course, like the GM and Q Learning. I have individual courses for each of these models which do go in depth and I don’t intend to repeat that material here. Furthermore, it would be impractical for you to have to learn a whole courses worth of material. For these reasons, the methods we are about to discuss will not be investigated in depth, but you should get a high level intuition for how they work. As always, if you’re interested in learning more, please ask me for references on the Q&A or through my website. Finally, note that the math parts of this lecture are optional. The goal isn’t for you to have a strong grasp of the math, but just a high level understanding of what we want to do. The goal is to get to the end of this lecture where I show you the code for how we will use an HMO. Since the code is very sure, I wanted to give you some idea of what that code is actually doing. So, again, focus on the high level principles over a deep understanding of the math. OK, so task number one, finding P of X in this case, X is the entire vector of observations, X one all the way up to 60. And remember that in our case, these represent stock returns. It sounds pretty simple, but in fact, this can get quite complex. OK, so how do we even begin with solving this task? Well, remember that we are equipped with a graphical model. Let’s suppose all I had was a single time step sequence. That is the one and X one. I know that X one depends on Z one. I also know that I have two probabilities. We have a Z one, which has its own distribution of Z. One, its all by itself, since there are no arrows going into it. We also have P of X1 given what we know that this should be a conditional distribution since X1 depends on Zwan. OK, so what can we do with this. Well in graphical models we are typically interested in the joint distribution. Luckily we can use the information we have to find it PVCs one Anzhi one is equal to X1 given Z one times Pazzi one. This is just the rule of conditional probability, but we know we only care about P of X, which in this case is just PVCs one. Luckily we know how to do this too. This is just marginalisation. In order to find PVCs one, we need to sum the joint distribution over all possible values of zwaan, as you recall. That’s just the sum from one up to M if we have M hidden states. OK, so that was just a very simple case, what if we have a sequence of length to now there are two extra variables Z to Annex two. However, we know how all these things are related and we now have some idea of what we want to find the joint distribution. So how do we find that joint distribution over these four variables, X1 and X2, Zwaan and Z2? Well, since one still has no arrows going into it, we can start with this one all by itself. Then we know that X1 depends only on one. So we can include P of x1 a given Z one as before, but we still have Z to an X two. We know that Z two also depends on as each one. So we have to multiply by Z to give Z one. Finally we have x2 which depends on Z two, so we have to multiply by P of X to give Z to. As before, we want to find P of X, which is now the joint distribution over the sequence X one, X two, this means that we have to marginalize the full joint distribution by something over Z one and Z two. So notice how we now have twosomes over I and J from one up to M. OK, so pretty easy so far. So if we keep following this pattern, what do we get? Well, let’s now assume that we have a sequence of length biggity. In this case, we can express the joint distribution over the sequences X and Z by following the pattern we had before. Notice that the X without any subscripts is the sequence X one up to 60 and Z without any subscripts. Is the sequence zwaan up to Z. As before, when we want to find P of X by itself, we have to sum over all the possible values of Z for all the. OK, so that’s how we find the probability of a sequence given a hidden Markov model. There’s one problem with the previous formulation, let’s consider how long it would take to compute. First, let’s look at the product, which is inside all the summations that’s approximately T multiplications. So we say it’s O of T, but now let’s consider the summations. We are doing a T joint summations, since we have to sum it using Z one all the way up to. However, each of those individual summations need to sum over m different possible values. So overall, the total number of things I need to sum is sometimes M, times M and so on T times. In other words, that’s M to the power T. So overall the complexity of this calculation is all of T times M to the power t. You may recognize that this is exponential growth. Luckily we do have algorithms to overcome this problem, which we won’t discuss at this moment. The important thing is to understand that we can do this calculation and why we want to do this calculation. The reason why we care to find PVCs is because, as you know, we need to find the parameters of our model. The parameters of our model consist of a and the Music sigmas for our Gaussians. So why does PVCs help us find the model parameters? Well, this is nothing but maximum likelihood estimation. This is the same thing we do when we want to find MU and Sigma for a Gaussian distribution. Let’s suppose we collect a bunch of heights from the people in our class and we would like to fit a Gaussian model, the MU and Sigma that we find should be the MEU and Sigma that maximize P of X. The joint distribution of the data we collected, for example, we would not want to set to be 1000 feet since that would make X very small. Clearly the average height of the people in our class is not 1000 feet. Similarly, for an HMO, we need P of X because that’s what we want to maximize in order to find a the muse and the Sigma’s. So how do we maximize effects in order to find a the muse and the Sigma’s? Well, there’s actually more than one answer to this question, but here are two method. Number one is to use the classic general method of gradient descent. Let’s suppose that all the parameters are stored in some big vector called theta. Then we find the gradient of the log of P of X with respect to Theta. And then we take small steps in that direction. After doing this several times, we arrive at the maximum. Note that just like maximizing the likelihood of a Gaussian, it’s much easier to work with the log of P of X compared to the original P of X. If you would like some references on why gradient assassin works, please ask me about it on the Q&A method. Number two is to use a more advanced method called expectation maximization. This is an algorithm designed specifically for probabilistic models with hidden variables that you need to sum over. Another example is the GME, which we saw earlier. The details of expectation maximization are outside the scope of this course, but if you want to learn more, you’re encouraged to ask about it on the Q&A. Oh, and by the way, this is task number two for any human. So task number one is to find P of X. Task number two is to maximize P of X so that we can find the parameters of the HMO. And that’s a the muse and the Sigma’s. But note that task number one can still be useful on its own. For example, if we have a sequence and we want to know whether or not it’s probable under our model, we can check the value of P of X. If we have several models for different classes, we can check which model gives us the largest P of X to determine which class that sequence came from. For example, we might want to classify stocks as healthy or poor or good investments and bad investments, so that would be how we can use ACMS to build sequence classifiers. Task number three is called decoding, also known as inference, as you recall, our graphical model shows Z pointing to X, but in the physical world, what we actually have is X the observations and we do not know Z. So decoding takes us in the opposite direction. It says here is X. Now give me back Z. The intuition is that we want to choose a sequence for Xie xie one up to Zeti such that the probability of our observations is maximized. So we see that task number one appears again since we are again required to find the probability of a sequence of observations. So let’s think about how we might perform this task from a naive standpoint, so we have our observations X1 up to 60. We want to find the best settings of zwaan up to ZT such that the probability of observing X1 up to 60 is maximized as an example of some possible solutions. Consider the sequence is low, low, low and so on, or high, high, high and so on. That is sequences of all those or sequences of ohis. Clearly these two sequences of these are probably not the answer. The answer is most likely a combination of lows and highs. For example, consider the following sequence of stock returns. I’ve highlighted the periods of high variance and the periods of low variance. Clearly, I would like my HMO to return high during the periods of high variance and low during the periods of low variance. So that should give me a larger probability. To be more precise, the exact probability we want to maximize is of given X, that is to say, the probability of hit in states, given some observations, this makes sense since the observations are actually what we are given. However, notice that P of Z given X is just X, Anzhi divided by P of X. This is just the rule of conditional probability. Once again, of course, we know how to find both of these from task number one. Furthermore, since we are taking the max over all possible values of Z, this actually doesn’t depend on P of X at all. Therefore, this is the same as maximizing over the joint distribution P of X and Z. Now that we know this, the rest is easy. OK, so let’s say we want to find the best possible sequence, zwaan up to Zeti that maximizers PVCs and Z. In order to do this, let’s simply generate all possible sequences for zie. Then we can loop through each of these sequences, calculate PVCs, ANZ, and then keep the one that gives us the maximum value. OK, so that should seem pretty straightforward. Please walk through the pseudocode to make sure that it makes sense to you. Again, I want to reiterate that this is for intuition only, as you recall, there is still an end to the power t possible sequences. So in fact, this has exponential complexity. We do have a more efficient way of decoding, which is called Viterbi decoding. This is named after the person who invented the algorithm. Andrew of Iturbi. He is also the co-founder of Qualcomm, which you may have heard makes chips that power many of today’s smartphones. So I hope that this lecture provided you with some insight into what an HMO does under the hood when we use it in practice, using an HMO is actually very simple because there are libraries out there which have already implemented the algorithms we discussed. Note that unlike many other machine learning models, the HMO is not part of Cycad Learn. One library I really like is called Amla, which is available for Python. This is not automatically installed with Google CoLab, so we’ll need to install it using PIP. So here’s the pseudocode for using an HMO, as you’ll see, even though it’s not part of Saika learn, the way that it works pretty much follows the usual patterns. First, we start by instantiating and Shamim objects. We use Gaussian emem because our model has Gaussian observations. Not that the usual HMO students learn about is one with discrete observations. Gaussians are more useful for us because they handle continuous data. Next, we call model fit to train our model parameters, as you recall, this sets A and the muse and the Sigma’s of each Gaussian. Finally, when we want to make an inference, we call model that predicts and this gives us back Z, the most likely sequence of hidden states internally. This runs of Iturbi.

5. HMM for Modeling Volatility Clustering in Code

In this lecture, we are going to implement what we’ve learned in the previous lectures, that is to say, we’re going to use a hidden Markov model to model a sequence of stock returns and find their low and high volatility regimes. This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise, which I always recommend is once you know how this is done, to try and recreate it yourself with as few references as possible. As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth discussion. If there’s anything in this lecture you didn’t understand or you think I missed a step or didn’t explain why we were doing something, please use the Q&A to inquire. As usual, you can look at the title of the notebook to determine what notebook we are currently looking at. OK, so as mentioned, the library, Miller is not included in Google CoLab, however, it’s very easy to install using PIP, so that’s what we’ll do. Next, we import the HMS module from the HMS Learn package. Next, we download the spy data set. Note that this analysis could work for UNIVARIATE or multivariate time series. However, I think UNIVARIATE Time series is more clear. As always, you’re encouraged to try out a different data set as an exercise. Next, we import pendas, numpty and matplotlib. Next, we load our data set into a data frame by calling pedigreed CSFI. Next, we call the head to remind ourselves about what is inside our data frame. As you can see, it’s just the standard open, high, low, close and volume columns. Next, we calculate the log returns by taking the log of the close column and taking the first difference, we assign this to a new object called Returns. Next, we drop any missing values by calling dropping a. Next, we plot a histogram of our returns to remind ourselves that what we are working with is a fat tail distribution. It is tall and skinny, unlike a Gaussian. Next, we create an A.M. objects in our case, we would like a Gaussian. Hmm. Because our observation distribution is a Gaussian note that we pass in some arguments to the constructor. First, we have N components, which refers to the number of hidden states in our case because we know what we are trying to model. The number of states is easy to choose in other applications. You might use other hyper parameter selection methods like the A.B.C. or cross-validation. We also set the covariance type two diag, although this is not necessary in our case. The reason we want to do this is a bit outside the scope of this course. So you can ignore my explanation if you wish. OK, so as mentioned previously, in general we can have a multivariate time series. Of course, if our TIME series is multivariate, then the Gaussian distribution will also be multivariate. And as you know, the multivariate Gaussian distribution has a mean vector and a covariance matrix. This is opposed to the univariate Gaussian, which just has a single scalar value for the mean and the single scalar value for the variance. Now, if you look up the formula for the multivariate Gaussian, you will notice that it includes the inverse and determinant of the covariance matrix. These are analogous to the univariate Gaussian when we divide by the variance. So what happens? Well, there is a scenario where all your data points can be clustered very close together, which results in what is called a singular covariance matrix. Basically, the scalar analogue of this would be when you have a variance equal to zero, as you know, dividing by zero leads to numerical stability issues. Therefore, we would like our variance to not be zero and we would like our covariance to not be singular. One way to avoid this is by using a diagonal covariance, which means that instead of having a full dividing matrix, we only have the individual variances for each of the dimensions. That is to say, the covariance matrix is diagonal, meaning that all the off diagonal values are zero. This, in turn, it makes it much easier to calculate the inverse and the determinant. In any case, this is not super relevant to this code, since we are currently using only a univariate time series. And so there is no covariance matrix internally. However, H.M. Learn is using a multivariate Gaussian. It just happens to be a very common and sensible choice to use the diagonal covariance option. OK, so next, we create our data set, which is easier to work with as an umpire. So we take the returns call to numpty, which converts it to an umpire, and then we reshape it to be an end by one matrix. As you recall, in machine learning, we generally work with two dimensional data sets where the last dimension is the feature dimension. This should be simple to understand at this point, since we’ve seen other models in cycle learn which behave the same way. Next, we call Model Duffett, which finds the parameters of our model. Next, we do inference with our model by calling model does predict X, as you recall, this runs of Iturbi and returns the most likely sequence of hidden states. So if we print out the states, we see a bunch of ones and zeros in an array. So what does this mean? Remember that we are doing unsupervised learning. Therefore, the states inherently have no meaning and it’s up to us to assign meaning from what we observe. This is just like models like K means clustering. When you do clustering, your clusters are just 012 and so forth. You don’t inherently know what the clusters mean. You have to look at the data. Only by looking at the data can you see things like cluster zero mean small, cluster one means big and so forth. OK, so how can we assign meaning to these states? Well, one idea is to simply plot the sequence of these as if it were a time series that is a time series of zeros and ones. We can plot this along with the original stock returns to see if there’s any pattern. So we start by calling plotted out subplots and making a plot of size 10 by five. Next, we call plotted out supply to one one and we plot Z. This makes a two by one grid and we plot Z in the first position. Next, we call alternate supply two one, two, and we plot the returns. This means that we plot the returns in the same two by one grid in the second position. OK, so let’s look at the result. So we can see that there is, in fact, some kind of pattern when we have large values in the returns, we tend to see thicker lines on the ZIS when we have small values in the returns. The these look pretty sparse. OK, so this is an encouraging result. It tells us that our model can in fact tell the difference between the two regimes. It’s important to remember, however, that the chart on the top is not shaded in that is dark, does not mean high, whereas light means low. Remember that the state values can be zero or one. So as an exercise for now, think about what it really means for an area to be dark or shaded in. I’ll explain this later in the lecture. But for now it would be a good exercise to consider this on your own. OK, so in the next step in the script, which I think will be helpful to visualize our results, we will color code the Time series instead of just plotting the states themselves as a time series. That is, we’re going to plot the returns. But when we do so, we’re going to make the values that belong to one state, one color and the values that belong to the other state, a different color. So let’s start by creating to erase that only contain nans. Recall that when we plot Nans, they simply don’t appear on the plot, which is what we want. Then we’re going to fill in the values corresponding to the state the values belong to. That is, all the values belonging to State Zero will go in one array and all the values belonging to state one will go in the other array. Then when we plot them at the same time, only one of these will have a value at each time step and the other will be Enan. This will result in each of the states being a different color. OK, so first we’re going to create two arrays called Returns Zero and returns one. Of course, these both have to be the same size as Z. Next, we assign and again to each of the arrays for each value. Next, we fill in returns zero, but only for the time, steps where the state actually is zero. To do that, we can index return zero. Using Z equals equals zero. As you recall, Z equals equals zero returns a boolean array of trues and falses. You can try this in your terminal if you don’t believe me. Of course it will be true when XYZ is actually zero and false. Otherwise, when we index return zero with this boolean array of trues and falses, we get back only the values where the index is true. Again, you can try that yourself in the terminal on the right side. We do the same thing, but with the original returns. So in conclusion, what does this do? It assigns the returns from the original array to the new array return zero, but only on the days when the state is equal to zero. Next, we do the same thing, but for returns one and where the state must be one. Therefore, whenever returns zero has a value returns one is then again and whenever it returns, one has a value return. Zero is then n next we apply return zero and returns one. We also call p.l.c. the legend so we know which coloristic zero and which is state one. OK, so let’s run this. So we see that blue values correspond to state zero, whereas the orange values correspond to state one. It’s also easy to assign meaning to these states based on what we see. Clearly, the orange values are larger and the blue values are smaller. Therefore, we would say that state zero corresponds to low volatility and state one corresponds to high volatility. However, notice that the values do seem to jump around a lot, especially when we are in state one, when we are in state one, sometimes we don’t stay in state one for very long and sometimes when we’re in state one, we alternate quite a bit between blue and orange. Now we have to ask, is this the behavior that we want to see? Does it make sense if the regime changes every single day? That’s probably not the case since we want volatility clustering, not volatility fluctuations. In the next step, we’re going to look at the state transition matrix, which will provide more insight. This can be done by calling the transmat attribute of our model. So these results should make sense, given what we’ve learned first we can see that the probability of going from State Zero back to itself, which is located at the index zero zero, is about 75 percent. This is the largest number in this matrix. That makes sense because the longest runs we see in our time series of stock returns is when the stock returns stay in the low volatility state. That is, the probability of going from low to low should be large. Now, let’s consider the probabilities in the second row, the probability at index one zero is the probability of going from state one to state zero. The probability at index one one is the probability of going from state one, a two state one that is from high to low and from high to high. Both of these are approximately 50 percent. So does that make sense? It does make sense because that’s what we were seeing earlier. We saw that when the state goes high, it often switches back to low very quickly. That is, we have an even chance of going from high back to high or from high to low. So this makes sense for two reasons. First, when we look at our Time series, we see that the periods where the state is high are generally very short. Sometimes it seems like it’s just a single value. Furthermore, this makes sense in the context of the plot we saw earlier where we plotted Z as a time series in this plot. We saw that the thick areas corresponded to when the returns were large. Now it makes sense why these lines are so thick. It is because they correspond to going back and forth between zero and one very quickly. In other words, the reason why this line appears thick is because we are zoomed out so far that all we can see is a blue area. But in fact, if we zoomed in, we would see the state rapidly moving from zero to one and vice versa. This is opposed to when we are in the low state where the value just stays at zero. OK, so this makes sense in terms of our Gaussian mixture model, basically the high value returns belong to the high state and the smaller returns belong to the low state. These can change rapidly, as you saw. This makes sense just because today’s return was very positive or very negative, this doesn’t guarantee that the returns will be just as large tomorrow and the next day and so forth. Instead, what we might expect to see is that we have a lot of very positive or very negative values over a period of many days. But this does not mean that the returns will be very positive or very negative every day. So I hope you understand the difference. To be in a high volatility regime does not require that the return is very large every single day. It only means that the return is large overall for many of those days. In other words, if we really want to detect regimes, perhaps we don’t want the states to switch so often. In other words, we don’t want to see high, low, high, low, high, low, like what we are seeing here. Another way of saying this is that it might make more sense if the high volatility regime lasted longer and allowed for a few smaller returns. So what’s one way we can model this, one thing we can try to do is to set the state transition matrix manually, as mentioned in the theory lectures, you can also try to label the data yourself and then calculate the transition probabilities using counting. In any case, what we are doing now is just the test to help us understand the behavior of an HMO. So what were you going to do is set the state transition probability of staying in the same state very high for both states. We would expect that by doing this, we will make switching states so unlikely that it won’t happen very often. OK, so now the probability of going from high to high is ninety nine point nine percent and the probability of going from low to low is ninety nine point nine nine percent. The next step is again to call, Mortada predicts, to retrieve the most likely sequence of hidden states. Next, we create the same plot as before, what we plot the hidden state sequence as a time series directly on top of the original stock return time series. So now we see what we expect now, there are very few transitions between states and the model prefers for the state to remain the same. Also, it’s clear that the model does a good job of identifying the difference between large returns and small returns. In this case, state one refers to the high volatility regime and state zero refers to the low volatility regime. Remember that state zero in state one and don’t have any meaning until we assign the meaning, Z would be just as valid if the zeros and ones were reversed. OK, so this makes sense by changing the state transition matrix to have more extreme probabilities, we make it so that switching between different states is more improbable and this makes the sequence of states more stable and more plausible. Next, we draw the subsequent plot that we had before, where we color coded the time series of stock returns according to their stay. Since I’ve already explained this code, I won’t do so again. So what do we see? So I think this plot looks a lot better than what we had before. This is another way to confirm that the states are more stable. And it’s also clear from this plot what we would call a high volatility regime and a low volatility regime. Clearly, being in one state leads to much larger returns overall and being in the other state leads to much smaller returns.

srt_to_txt