1
00:00:11,690 --> 00:00:17,040
In this section of the course we are going to introduce the idea of artificial known that works or eons

2
00:00:17,150 --> 00:00:18,310
for short.

3
00:00:18,530 --> 00:00:23,690
Specifically we're going to talk about a certain kind of artificial known that work called a feed forward

4
00:00:23,690 --> 00:00:25,060
neural network.

5
00:00:25,070 --> 00:00:30,560
This is the most basic kind of neuron that work but as you'll see the concepts involved go quite deep

6
00:00:30,860 --> 00:00:36,140
and they also form the basis for other kinds of neural networks such as convolution on neuron that works

7
00:00:36,200 --> 00:00:37,710
and recurrent or networks

8
00:00:42,910 --> 00:00:44,080
to begin this discussion.

9
00:00:44,110 --> 00:00:48,450
Let's start with why and how neural networks came to be in the first place.

10
00:00:49,680 --> 00:00:53,970
This is something that a lot of teachers skip over but it's something that I find quite interesting

11
00:00:53,970 --> 00:00:55,020
and inspirational.

12
00:00:55,590 --> 00:01:00,900
If you think Neural Networks are a cool new model to help you pick stocks or play Mario Kart then you

13
00:01:00,900 --> 00:01:02,970
are still looking for the small fish.

14
00:01:03,120 --> 00:01:05,760
In fact at networks are way more interesting than that

15
00:01:10,970 --> 00:01:15,980
as you may have realized that the name artificial known that work means that we are trying to artificially

16
00:01:15,980 --> 00:01:19,010
create a neural network in a computer.

17
00:01:19,070 --> 00:01:24,860
OK so then what's a real name on that work as in a non artificial neural network the name neuron that

18
00:01:24,860 --> 00:01:30,790
network derives from neurons which are the cells in your brain and extend throughout your nervous system.

19
00:01:30,800 --> 00:01:36,770
Now this is probably too obvious for most of you but let's state it anyway just in case your brain is

20
00:01:36,770 --> 00:01:38,410
what you used to think.

21
00:01:38,660 --> 00:01:43,970
Neurons in your brain are connected to each other and can communicate with each other via electrical

22
00:01:43,970 --> 00:01:46,540
and chemical signals.

23
00:01:46,580 --> 00:01:51,960
Amazingly this simple physical and chemical system is what makes you you.

24
00:01:52,250 --> 00:01:57,570
All of your thoughts and aspirations your emotions and every action you take throughout the day.

25
00:01:57,650 --> 00:02:03,920
It's all driven by your neurons which in turn are just sending electrical and chemical signals around

26
00:02:03,950 --> 00:02:05,120
amongst themselves

27
00:02:10,340 --> 00:02:14,780
once scientists realize what the brain was doing and what it was responsible for.

28
00:02:14,780 --> 00:02:19,970
The next question was almost obvious in hindsight can we build a brain.

29
00:02:19,970 --> 00:02:24,830
I mean if the brain is just a network of neurons and we can simulate neurons in a computer.

30
00:02:25,400 --> 00:02:30,620
If we connect a bunch of neurons through a computer simulation it seems that it might be possible to

31
00:02:30,620 --> 00:02:34,460
create some form of intelligence and artificial intelligence.

32
00:02:34,460 --> 00:02:35,090
You might say

33
00:02:40,260 --> 00:02:44,550
so let's take our model of a single neuron logistic regression.

34
00:02:44,580 --> 00:02:49,920
Now let's imagine that we have multiple neurons all taking in the same inputs but computing something

35
00:02:49,920 --> 00:02:51,630
different now.

36
00:02:51,660 --> 00:02:53,220
Let's do it again.

37
00:02:53,250 --> 00:02:56,910
Now we have multiple logistic regressions.

38
00:02:56,910 --> 00:03:00,990
Now let's imagine that all these neurons are connected to more neurons.

39
00:03:00,990 --> 00:03:07,840
So we just repeat the process pretending that the new layer of neurons are actually inputs to more neurons.

40
00:03:07,860 --> 00:03:13,230
That's basically a very tiny model of the brain neurons connected to neurons.

41
00:03:13,230 --> 00:03:16,110
Of course this is necessarily simplistic.

42
00:03:16,170 --> 00:03:19,470
One side is the input and one side is the output.

43
00:03:19,470 --> 00:03:22,230
Of course the actual brain is much more complex.

44
00:03:22,230 --> 00:03:25,890
There are many inputs and many outputs in the middle.

45
00:03:25,890 --> 00:03:31,600
Wires can crisscross if we have a later neuron connecting back to an earlier neuron.

46
00:03:31,650 --> 00:03:34,050
We call that a recurrent connection.

47
00:03:34,050 --> 00:03:39,720
The neural networks we are about to discuss in this section contained no such complexities.

48
00:03:39,870 --> 00:03:45,540
Instead because the input is on one side and the output is on the other side and we go from input to

49
00:03:45,540 --> 00:03:50,220
output in a layer wise fashion we call this a feed forward neural network

50
00:03:55,440 --> 00:03:56,580
in the rest of this lecture.

51
00:03:56,580 --> 00:04:00,900
We are going to outline what we will discuss in this section of the course.

52
00:04:00,990 --> 00:04:05,550
First we are going to start out again by discussing the model architecture.

53
00:04:05,550 --> 00:04:11,040
As you know the model we'll be discussing in this section is the feed forward neuron that we're the

54
00:04:11,040 --> 00:04:16,520
next step after discussing the model architecture will be to go back to the geometric picture.

55
00:04:16,800 --> 00:04:21,990
If you recall my model goes machine learning is nothing but a geometry problem.

56
00:04:22,050 --> 00:04:28,440
So how do neural networks extend the capabilities of a basic linear model in terms of solving this geometry

57
00:04:28,440 --> 00:04:30,000
problem.

58
00:04:30,000 --> 00:04:37,230
Next we're going to go more in-depth and discuss activation functions activation functions are very

59
00:04:37,230 --> 00:04:38,490
important in neural networks.

60
00:04:38,520 --> 00:04:43,890
They are what make a neural networks more expressive than the simple linear models you saw in the previous

61
00:04:43,890 --> 00:04:45,590
section.

62
00:04:45,600 --> 00:04:51,670
After that we're going to discuss how to do multi class classification using neural networks.

63
00:04:51,780 --> 00:04:56,610
If you recall in the previous section we only discussed the binary classification.

64
00:04:56,610 --> 00:05:02,970
This works if we only have two classes dog or cat fraud or no fraud purchase or leave the store and

65
00:05:02,970 --> 00:05:03,420
so on.

66
00:05:04,080 --> 00:05:09,840
But what if we have more classes for example we might be working on a self-driving car that needs to

67
00:05:09,840 --> 00:05:13,850
be able to recognize multiple kinds of objects on the road.

68
00:05:13,980 --> 00:05:20,780
In this case a binary classification is not good enough and we need multi class classification.

69
00:05:20,790 --> 00:05:28,150
Next we'll switch gears from talking about the known that work itself to a new kind of data images Deep

70
00:05:28,150 --> 00:05:32,830
Learning excels on unstructured data such as images text and sound.

71
00:05:32,830 --> 00:05:38,050
We'll look at how images are represented in a computer so that we can get back to the situation where

72
00:05:38,050 --> 00:05:41,110
we can say all data is the same.

73
00:05:41,110 --> 00:05:46,810
Finally we'll look at some notebooks that demonstrate how to do image classification using a neural

74
00:05:46,810 --> 00:05:49,260
network for multiple classes.

75
00:05:49,270 --> 00:05:54,070
We'll also look at noon that works for regression and see how they can be much more expressive than

76
00:05:54,070 --> 00:05:55,420
a simple linear model.