1
00:00:00,780 --> 00:00:08,550
Lesson nine project for people counting with ultrasonic sensor, specifically for model training and

2
00:00:08,550 --> 00:00:15,270
development as compared to fully connected layers, convolutional layers save a lot of time and money.

3
00:00:15,670 --> 00:00:18,420
However, when the amount of data grows.

4
00:00:18,870 --> 00:00:25,710
For example, when we do inference on a high resolution image or longer time series data training,

5
00:00:25,710 --> 00:00:30,840
a deep convolutional network still takes a lot of computing power.

6
00:00:31,260 --> 00:00:37,620
Another problem with models that only use convolutional layers is that during training, the model also

7
00:00:37,620 --> 00:00:41,460
learns the location of features, which is not what we want.

8
00:00:42,000 --> 00:00:43,230
A cat is still a cat.

9
00:00:44,010 --> 00:00:50,820
No matter where it is in the image, we can apply downsampling during model training to solve both problems

10
00:00:50,820 --> 00:00:57,570
at once, which will reduce the number of connections and the size of the model and let the model learn

11
00:00:58,110 --> 00:01:01,530
the features regardless of where they are in the data.

12
00:01:02,010 --> 00:01:09,330
We can add pooling layers between convolutional layers, max pooling, mean pooling and average cooling.

13
00:01:10,320 --> 00:01:16,290
Pooling is the process of choosing pooling operation, which is like a filter for feature maps because

14
00:01:16,980 --> 00:01:21,090
the pooling operations and filters are smaller than the feature map.

15
00:01:21,180 --> 00:01:27,390
They are almost 22 pixels applied with a strike of two pixels, which is about the same size as the

16
00:01:27,390 --> 00:01:27,900
feature map.

17
00:01:28,170 --> 00:01:34,500
So basically, this means that the pooling layer is going to cut the size of each feature map by a factor

18
00:01:34,500 --> 00:01:35,040
of two.

19
00:01:35,400 --> 00:01:40,560
For example, the number of pixels or values will be cut in half for each feature map.

20
00:01:41,190 --> 00:01:49,690
The result of applying a layer called bullying to a feature map of four by four or 16 pixels will be

21
00:01:49,710 --> 00:01:55,620
pulled feature map with two by two steel 16 pixels as the result of four pixels.

22
00:01:57,090 --> 00:02:00,150
See how the average bullying operation works on four by four.

23
00:02:00,150 --> 00:02:07,830
Matrix When using convolutional layers in edge impulse, iMac's pooling layer is automatically applied

24
00:02:07,830 --> 00:02:09,300
after convolutional layer.

25
00:02:09,720 --> 00:02:13,680
You can delete it or change its parameters if switching to expect nodes.

26
00:02:14,070 --> 00:02:21,200
When training a model, think for expansion, but make sure you have lifeI libraries installed in Arkadelphia

27
00:02:21,210 --> 00:02:21,330
trim.

28
00:02:21,330 --> 00:02:24,270
Where is the latest version of Real Terminal?

29
00:02:24,600 --> 00:02:28,080
You also need to download the following libraries from the GitHub.

30
00:02:29,010 --> 00:02:32,250
The following links are here and the slides you may just copy.

31
00:02:32,250 --> 00:02:34,110
Paste them or click on that.

32
00:02:34,750 --> 00:02:41,610
Then make sure to install the file libraries using Arduino Library Manager for Pub Sub Client and message

33
00:02:41,610 --> 00:02:42,000
back.

34
00:02:43,050 --> 00:02:48,310
Now for Practice Mode When you're done with data collection, create your impulse set window length

35
00:02:48,330 --> 00:02:54,420
to one thousand five hundred millisecond and window size increase to 500 milliseconds.

36
00:02:55,380 --> 00:03:02,730
1500 millisecond is more than enough time for a person to walk in or out of a door unless they move

37
00:03:02,730 --> 00:03:03,540
very slowly.

38
00:03:03,930 --> 00:03:10,920
During this type, we only have two blocks to play with real data or spectral analysis, both of which

39
00:03:10,920 --> 00:03:11,660
we can try.

40
00:03:12,150 --> 00:03:18,000
Flatten blocks will remove all of the time domain information from the data, so we won't use it to

41
00:03:18,000 --> 00:03:19,590
figure out where we are going.

42
00:03:20,220 --> 00:03:27,960
Applying a fast for your transformed data samples The spectral analysis block moves from signal from

43
00:03:27,960 --> 00:03:30,210
the time domain into the frequency domain.

44
00:03:30,600 --> 00:03:35,430
With me use the FFO, we don't have to worry about the frequency of the signal.

45
00:03:36,000 --> 00:03:41,310
We can't tell if a person is coming in or leaving the room based on the frequency of the signal.

46
00:03:41,910 --> 00:03:46,500
You can see this in the data visualization after the spectral analysis block.

47
00:03:47,310 --> 00:03:52,650
It's hard to tell in and out data samples from each other actually changing the process.

48
00:03:52,650 --> 00:03:54,690
Napoli spectrogram doesn't really help.

49
00:03:55,380 --> 00:04:02,310
The best we could get was seventy nine point six, but there was a lot of confusion between in and out.

50
00:04:03,240 --> 00:04:10,440
In this case, the winner is raw data based on scale, plus one the convolutional network again as before.

51
00:04:10,920 --> 00:04:17,580
You will need to switch to expert mode and change all one strikes to one and pulsates to four.

52
00:04:18,000 --> 00:04:20,700
This will help you get ninety two percent of the time, right?

53
00:04:21,180 --> 00:04:24,480
Do you think ninety two percent accuracy is already good enough?

54
00:04:24,960 --> 00:04:27,450
If not, what can be done to improve it?

55
00:04:28,230 --> 00:04:35,820
Ninety two percent is fairly good as proof of concept or prototype, but horrible as a production model

56
00:04:36,330 --> 00:04:39,190
for production, mileage may vary.

57
00:04:39,210 --> 00:04:45,240
If your application is critical in somehow using an automated control and decision making, you don't

58
00:04:45,240 --> 00:04:49,620
really want to have anything below 98 to 99 percent, right?

59
00:04:50,010 --> 00:04:51,420
And even that might be low.

60
00:04:51,850 --> 00:04:56,250
Think about something like a face recognition system for payment or authentication.

61
00:04:57,000 --> 00:04:59,670
Do you think there are ways to improve that?

62
00:04:59,790 --> 00:05:08,070
You see of this system, ultrasonic sensor is cheap and ubiquitous sensor, but it is relatively slow

63
00:05:08,070 --> 00:05:09,360
and not very precise.

64
00:05:09,900 --> 00:05:18,120
We can get better idea by using the Grilles ATF mini lidar module, get more data and possibly at least

65
00:05:18,150 --> 00:05:23,970
a sensor lower at normal human waste level to make sure it can detect shorter than normal height people

66
00:05:23,970 --> 00:05:24,750
and children.

67
00:05:25,920 --> 00:05:27,240
Two are better than one.

68
00:05:27,720 --> 00:05:32,910
Having two sensors taking measurements at slightly different places will not add too much data.

69
00:05:33,180 --> 00:05:40,050
We only have one data point in each sample, but might increase the accuracy to explore more interesting

70
00:05:40,050 --> 00:05:40,650
ideas.

71
00:05:41,130 --> 00:05:46,170
A built in light sensor can be used if we are terminal is appropriately located.

72
00:05:46,830 --> 00:05:53,910
Now, once the model is trained, we can perform life classification with data from device here.

73
00:05:53,940 --> 00:06:01,530
We found that with the size increase of 500 millisecond actually doesn't work that well and produces

74
00:06:01,530 --> 00:06:02,880
more false positives.

75
00:06:03,300 --> 00:06:10,530
So the next step when deploying to the device, it is better to increase the value to 750 millisecond

76
00:06:10,530 --> 00:06:14,130
and to deploy the model to real terminal.

77
00:06:14,430 --> 00:06:16,080
Make sure to go to a deployment tab.

78
00:06:16,440 --> 00:06:24,150
Choose the Arduino Library downloaded, extract the archive and put it inside of your Adrenals Libraries

79
00:06:24,150 --> 00:06:24,690
folder.

80
00:06:25,200 --> 00:06:31,620
The next this time, we will be using continuous inference example to make sure we are not using any

81
00:06:31,620 --> 00:06:32,520
important data.

82
00:06:33,090 --> 00:06:38,400
If you remember in the first lesson the course for the inference we would collect all the data points

83
00:06:38,400 --> 00:06:39,210
in the sample, right?

84
00:06:39,930 --> 00:06:43,050
Perform the inference and then go back to sampling.

85
00:06:43,500 --> 00:06:49,080
That means when feeding the data to a neural network, we will post a data collection and lose some

86
00:06:49,080 --> 00:06:49,710
of the data.

87
00:06:51,300 --> 00:06:59,130
Now that is not optimal, we can use either dMY the direct memory, access, threading or multiprocessing

88
00:06:59,130 --> 00:07:00,210
to fix this issue.

89
00:07:00,690 --> 00:07:07,770
In this case, we will use the free Artiols and threads because the real terminal MCU or the cortex

90
00:07:07,770 --> 00:07:10,680
and 4-F core has only one core.

91
00:07:11,250 --> 00:07:17,850
People who use the free artless are going to stop friends for a short time, get the data sample and

92
00:07:17,850 --> 00:07:23,310
then go back to inference again, which is what is going to happen this way.

93
00:07:23,460 --> 00:07:29,220
The real inference will take a little longer, but difference isn't that big for the use case.

94
00:07:29,910 --> 00:07:36,810
Every 500 millisecond we do in stress, so every 500 milliseconds slice off the time we know we'll be

95
00:07:36,810 --> 00:07:38,880
done in France on three times.

96
00:07:39,480 --> 00:07:40,530
This is how it works.

97
00:07:41,070 --> 00:07:46,620
We look at three inferences and we choose the one with the most confidence.

98
00:07:46,830 --> 00:07:54,420
For example, if we have the most confidence in a label that reads out, we'll use that label to make

99
00:07:54,420 --> 00:07:55,560
the testing easier.

100
00:07:55,860 --> 00:08:02,460
We will add the lines that turn on the real terminal screen when someone comes into the room and turn

101
00:08:02,580 --> 00:08:04,440
it off when someone lives.

102
00:08:05,220 --> 00:08:13,200
Now, open examples search for name of your project and then look for nano underscore.

103
00:08:13,200 --> 00:08:21,780
B 33 underscore a sense underscore accelerometer, underscore continuous sketch and replace everything,

104
00:08:21,780 --> 00:08:28,950
including the run inference background, which is the function declaration now above setup function,

105
00:08:28,950 --> 00:08:33,390
which is the following code block, as you can see on the side of the screen.

106
00:08:34,590 --> 00:08:41,670
And then in setup function, make sure to initialize the LCD screen and delete all the lines that are

107
00:08:41,670 --> 00:08:50,190
related to accelerometer, then in place of interference underscore threat, that story based the following

108
00:08:50,190 --> 00:08:50,850
code block.

109
00:08:51,270 --> 00:08:57,690
The reason we need to replace this line is because threat initialization is done differently and arguing

110
00:08:57,690 --> 00:09:00,720
obviously 33 cents and real terminology.

111
00:09:00,930 --> 00:09:01,560
That's why.

112
00:09:02,130 --> 00:09:07,920
So here so you can see we have already changed the following code on this slide.

113
00:09:09,850 --> 00:09:11,710
OK, now proceeding to the next line.

114
00:09:13,090 --> 00:09:17,050
Run interference, continuous function is largely unchanged.

115
00:09:17,410 --> 00:09:24,250
The only two things that needs to be change here are these following codes, so make sure you can copy

116
00:09:24,250 --> 00:09:24,520
them.

117
00:09:25,330 --> 00:09:31,990
Now the line above controls averaging or smoothing parameters that we apply to output of the model.

118
00:09:32,410 --> 00:09:37,960
You can experiment with the values to see what values allow for best performance in terms of true positives

119
00:09:38,260 --> 00:09:39,380
or false positives.

120
00:09:39,390 --> 00:09:39,700
Three.

121
00:09:40,360 --> 00:09:46,330
Well, in the original code, the data collection happens in loop function, or we determine all the

122
00:09:46,330 --> 00:09:48,040
free ideal exploit.

123
00:09:48,310 --> 00:09:52,480
It is better to implement data collection in a thread and leave the loop function empty.

124
00:09:53,080 --> 00:09:58,600
Delete the loop function in original code and replace it with the following code block.

125
00:09:59,590 --> 00:10:04,480
A sensor called an ultrasonic sensor is used to measure the distance between two objects.

126
00:10:05,050 --> 00:10:09,160
The sensor then copies the data to a buffer that can be used for free.

127
00:10:09,730 --> 00:10:16,270
Take note that because there are so many threads, taking too long to read here doesn't affect the whole

128
00:10:16,270 --> 00:10:16,810
system.

129
00:10:17,380 --> 00:10:21,760
It just temporarily pauses each thread until it's time to start another one.

130
00:10:22,240 --> 00:10:27,670
The free artioles can run other tasks, while the data collection thread is it running?

131
00:10:28,090 --> 00:10:29,470
No expansion task.

132
00:10:30,370 --> 00:10:37,180
Now the model works, but again, all in by itself, it's not suitable for actually applying it in the

133
00:10:37,180 --> 00:10:37,750
real world.

134
00:10:39,040 --> 00:10:44,410
Now, let's add two elements to make it into a full fledged application, which are a simple job.

135
00:10:44,440 --> 00:10:48,190
I need to upload the cloud with pretty graphs.

136
00:10:49,090 --> 00:10:56,860
We will use the LG G L Library for creating graphical interface and Microsoft Azure IoT Central Service

137
00:10:56,860 --> 00:11:00,100
for sending data to and visualization.

138
00:11:00,580 --> 00:11:07,810
Now, the resulting sketch is 693 lines long and has three concurrent threads run in our.

139
00:11:08,950 --> 00:11:13,570
Here is a quick recap of the steps you need to make it work with iOttie John.

140
00:11:13,960 --> 00:11:20,470
First is that you need to find a project in the course materials under the name Rio Terminal as the

141
00:11:20,470 --> 00:11:28,510
Central Dot is not, and then open it in Arduino IDG after the sketches uploaded and their configuration

142
00:11:28,510 --> 00:11:34,810
mode by pressing three buttons on top of the wheel terminal and resetting the device in configuration

143
00:11:34,810 --> 00:11:36,700
mode will be displayed on device screen.

144
00:11:37,300 --> 00:11:47,160
Connect to device with serial monitor with a board rate of 150 200 carriage return and set Wi-Fi Assist,

145
00:11:47,170 --> 00:11:53,680
ID password and Asura with essential credentials in the following format of this light.

146
00:11:54,070 --> 00:11:58,270
You can see here and then right after following this format.

147
00:11:58,840 --> 00:12:06,130
Follow this next steps Go to this website and then if you don't have a Microsoft account yet, make

148
00:12:06,130 --> 00:12:07,360
sure to register one.

149
00:12:08,110 --> 00:12:12,250
Go to Bill, look for a custom app and then enter the app name and unique.

150
00:12:12,250 --> 00:12:17,020
You are l choose free plan after an app is created.

151
00:12:17,260 --> 00:12:18,850
Go to the device templates.

152
00:12:19,510 --> 00:12:23,500
Make a new template of iottie device type, then choose a custom model.

153
00:12:23,860 --> 00:12:28,450
Add three capabilities as a below screenshot and two interfaces.

154
00:12:29,050 --> 00:12:30,520
Make sure to press and use that.

155
00:12:30,520 --> 00:12:36,460
Look for visualizing the device after finishing that and making sure everything is correct.

156
00:12:36,880 --> 00:12:38,470
You can now publish the template.

157
00:12:39,330 --> 00:12:46,030
Create a new device from template by going to devices, pressing the new remember to choose the template

158
00:12:46,030 --> 00:12:47,500
you just created and published.

159
00:12:48,100 --> 00:12:52,060
Get the ideas, quotes from administration and then proceed to device connection.

160
00:12:53,020 --> 00:12:55,000
Primary key from administration.

161
00:12:55,100 --> 00:12:58,180
Then look for device connection after that.

162
00:12:58,840 --> 00:13:06,820
Look for SAS IoT devices and device ideas from Devices tab where you just created your device on Step

163
00:13:06,820 --> 00:13:07,210
five.

164
00:13:07,600 --> 00:13:15,400
Here is the following result when following the steps that was mentioned earlier and then after configuration

165
00:13:15,400 --> 00:13:20,680
is successful, restart your terminal and it will start connecting to Azure IoT, etc..

166
00:13:21,190 --> 00:13:24,340
You can watch the detailed progress feedback on the serial terminal.

167
00:13:24,910 --> 00:13:32,650
You will then be able to see a device that was on dashboard that is changed to provision telemetry data

168
00:13:32,650 --> 00:13:36,100
from accelerometer sensor in device that received the raw data.

169
00:13:36,370 --> 00:13:43,210
Now when we finish, we add the parts that makes the edge impulse model ready and send telemetry functions

170
00:13:43,210 --> 00:13:43,540
work.

171
00:13:44,080 --> 00:13:50,260
We also change the descent telemetry function to send the number of people who came in left and a total

172
00:13:50,260 --> 00:13:52,180
of number of people in the room.

173
00:13:52,780 --> 00:13:59,110
We also add a simple UI with three buttons and a text field that shows information changes.

174
00:13:59,500 --> 00:14:04,870
You can see the sketch by opening Real Terminal and people counting as their central LV guide.

175
00:14:04,870 --> 00:14:07,310
That includes some other course materials.

176
00:14:07,360 --> 00:14:09,610
But don't forget the understudies, OK?

177
00:14:10,510 --> 00:14:12,250
The hardest part was making sure that.

178
00:14:13,140 --> 00:14:19,500
Work normally in each separate thread and did not affect other things, which I believe is a lot of

179
00:14:19,500 --> 00:14:19,860
work.

180
00:14:20,850 --> 00:14:24,940
The sacrifices had to be made so that the court didn't get too complicated.

181
00:14:24,960 --> 00:14:32,400
For example, the Alvin G l Dash app, the expulsion was put right after interface updates and not run

182
00:14:32,400 --> 00:14:33,120
every time.