1
00:00:11,060 --> 00:00:14,040
OK, so let's recap what kinds of problems we can solve.

2
00:00:14,060 --> 00:00:18,560
So far, we now know how to build a feed for neural network.

3
00:00:19,010 --> 00:00:21,650
This can handle two dimensional input arrays of size.

4
00:00:21,650 --> 00:00:29,630
And by these so in samples and features, in other words, just tabular data because of the rule, all

5
00:00:29,630 --> 00:00:30,710
data is the same.

6
00:00:30,920 --> 00:00:37,100
This can also handle univariate time series of shape and by t, just like we learned about in the machine

7
00:00:37,100 --> 00:00:37,960
learning section.

8
00:00:38,420 --> 00:00:42,140
And of course this applies to both classification and regression.

9
00:00:42,920 --> 00:00:47,380
The next question to consider is how we might handle multivariate time series.

10
00:00:47,780 --> 00:00:52,790
And this is because Human Activity Recognition is a multivariate time series.

11
00:00:57,360 --> 00:01:03,090
Now, one option would be to do the naive thing, suppose that we have a D dimensional time series and

12
00:01:03,090 --> 00:01:08,810
each time series has length T, so in effect, each sample has the shape TBD.

13
00:01:09,690 --> 00:01:17,190
Well, what we could do is simply flatten this into a single vector of size t times the then our feet

14
00:01:17,190 --> 00:01:24,390
for a neural network would take an inputs of shape and by T times d but this is not ideal since it treats

15
00:01:24,390 --> 00:01:25,540
every point equally.

16
00:01:26,100 --> 00:01:30,810
We know that there is a special structure, two time series that we might want to take advantage of.

17
00:01:35,440 --> 00:01:40,930
Suppose that we wanted to have a separate neuron for each dimension of the Time series, this would

18
00:01:40,930 --> 00:01:45,970
allow that single neuron to learn the temporal structure of that dimension's time series.

19
00:01:46,720 --> 00:01:49,450
This would give us some possibly useful feature vector.

20
00:01:50,230 --> 00:01:56,500
Then we could combine these feature vectors into a single large feature vector that represents the features

21
00:01:56,500 --> 00:01:57,900
for all the dimensions.

22
00:01:58,360 --> 00:02:01,310
We could then pass this through a final dense layer as usual.

23
00:02:02,440 --> 00:02:08,510
So because this model has multiple separate inputs, I refer to this as a multi tale's and neural network.

24
00:02:09,280 --> 00:02:12,430
You might wonder why this is not called a multiheaded neural network.

25
00:02:13,180 --> 00:02:18,510
The reason for this is we typically think of the output as the head or the top, and so by default,

26
00:02:18,510 --> 00:02:20,210
so the input becomes the tail.

27
00:02:20,590 --> 00:02:25,510
You can think of this as a kind of multi tailed beast, although I hear that this name has been taken.

28
00:02:30,180 --> 00:02:35,130
OK, so the next question to answer is, how do we build a multi tailored neural network?

29
00:02:35,760 --> 00:02:41,280
The main idea to keep in mind is there wherever you previously passed in a single input, it's also

30
00:02:41,280 --> 00:02:44,130
acceptable to pass out a list or tuple of inputs.

31
00:02:45,300 --> 00:02:49,470
Note that in other applications, it's possible to have multiple outputs as well.

32
00:02:49,650 --> 00:02:51,390
And this follows the same principle.

33
00:02:52,500 --> 00:02:56,970
In this class, we generally won't deal with multiple outputs, but it should be comforting to know

34
00:02:56,970 --> 00:02:58,300
that the API sensible.

35
00:02:59,760 --> 00:03:02,970
OK, so what should you do if you want multiple inputs?

36
00:03:03,810 --> 00:03:08,340
Well, you simply create multiple input objects for this example.

37
00:03:08,340 --> 00:03:10,130
Let's just say that D is equal to three.

38
00:03:10,350 --> 00:03:13,710
So we effectively have three time series each wavelength t.

39
00:03:18,450 --> 00:03:23,910
The next step is to create a dense layer for each of these inputs, so basically you can think of these

40
00:03:23,910 --> 00:03:26,310
as like three parallel neural networks.

41
00:03:27,000 --> 00:03:32,790
Now, technically, you could add more layers, but I leave that to you to try by yourself for this

42
00:03:32,790 --> 00:03:33,800
simple example.

43
00:03:33,810 --> 00:03:38,280
The next step will be to combine the output of these parallel neural networks together.

44
00:03:38,910 --> 00:03:44,470
Remember that you can think of neural networks as like transforming input vectors into feature vectors.

45
00:03:45,090 --> 00:03:50,400
So at this point, after passing each input time series through their own neural network, we now have

46
00:03:50,400 --> 00:03:53,340
a set of feature vectors for each of those Times series.

47
00:03:57,960 --> 00:04:04,470
So to combine each of these feature vectors together, we will use concatenation in a picture you can

48
00:04:04,470 --> 00:04:10,350
imagine simply lining up each of the feature vectors side by side to make one gigantic feature vector

49
00:04:10,980 --> 00:04:12,000
intenser flow.

50
00:04:12,030 --> 00:04:14,370
This can be done using the contaminate layer.

51
00:04:18,910 --> 00:04:24,520
Now, at this point, we have a feature vector which can be passed through yet more dense layers, in

52
00:04:24,520 --> 00:04:29,770
other words, through another neural network, for this example, we'll just have one layer to keep

53
00:04:29,770 --> 00:04:30,480
things simple.

54
00:04:31,300 --> 00:04:35,920
At this point, the number of outputs of the final layer should correspond to the task.

55
00:04:36,310 --> 00:04:40,960
So if you're doing multiclass classification with K classes, then you'll have key outputs.

56
00:04:45,600 --> 00:04:49,470
The next thing we should discuss in this lecture is how to create your model objects.

57
00:04:50,130 --> 00:04:56,490
Previously, when we only had one input, we instantiated a model object by passing in that single input

58
00:04:56,790 --> 00:05:00,780
along with the corresponding output when you have multiple inputs.

59
00:05:00,990 --> 00:05:04,620
You simply pass them into the first argument as a list or a tuple.

60
00:05:05,010 --> 00:05:07,020
Same goes if you had multiple outputs.

61
00:05:11,640 --> 00:05:16,200
Now, since your model has multiple inputs, this is going to change how you call a fit and predicts

62
00:05:16,200 --> 00:05:20,040
functions, but again, it follows the same pattern as before.

63
00:05:20,700 --> 00:05:27,390
So when you call a function, your first argument will now be a list of input arrays, since our multivariate

64
00:05:27,390 --> 00:05:28,860
time series will have the shape.

65
00:05:28,870 --> 00:05:35,370
And by TBD, if we index this in the final dimension, we'll get an array of shape and by T which are

66
00:05:35,370 --> 00:05:39,170
neural network will accept the same thing applies to predict.

67
00:05:39,870 --> 00:05:46,740
OK, so instead of a single input array, you now have a list of input arrays, specifically a list

68
00:05:46,740 --> 00:05:50,730
of length D where each element is an array of shape and by T.

69
00:05:55,440 --> 00:06:00,910
OK, so here's one last look at the multi tailed neural network for multivariate time series.

70
00:06:01,590 --> 00:06:05,000
Now, in practice, keep in mind that these can be pretty large.

71
00:06:05,370 --> 00:06:09,840
So you might want to use a for loop instead of trying to create each input one by one.

72
00:06:10,590 --> 00:06:13,710
Again, it's a pretty simple pattern to follow anywhere.

73
00:06:13,720 --> 00:06:15,880
You previously had a single input array.

74
00:06:16,080 --> 00:06:18,270
You now have a list of input arrays.