1
00:00:00,390 --> 00:00:06,690
In our previous lesson we made the drone detect faces in this lesson we will make the drone tracked

2
00:00:06,690 --> 00:00:10,340
the detected faces before we go into the codes.

3
00:00:10,350 --> 00:00:13,440
I want to explain you what we will be creating.

4
00:00:13,470 --> 00:00:21,120
Look at this image in the screen the outer screw will be the frame the x and y coordinates will be zero

5
00:00:21,180 --> 00:00:21,950
and zero.

6
00:00:22,140 --> 00:00:30,510
The width will be a maximum of 320 and height will be 240 the small square will be the area where the

7
00:00:30,510 --> 00:00:39,390
face is detected once the face is detected the x and y coordinates width and height can be calculated.

8
00:00:39,390 --> 00:00:46,110
Also the center point of the face can be calculated by dividing the width and height of the rectangle.

9
00:00:46,170 --> 00:00:48,600
Here I marked it with a red dot.

10
00:00:48,600 --> 00:00:52,340
The center point of the frame can be calculated as well.

11
00:00:52,740 --> 00:00:59,130
When the drone detects the face and determines the center of the face we will set the system to calculate

12
00:00:59,130 --> 00:01:03,360
the gap between the face Center and the frame center.

13
00:01:03,360 --> 00:01:09,270
Based on this gap the drone can fly to the point where the face is set to the center of the frame for

14
00:01:09,270 --> 00:01:13,200
example in this case the face is at a higher point.

15
00:01:13,200 --> 00:01:20,770
So the drone should fly up also the is on the left side so the drone should be flying to the left.

16
00:01:20,790 --> 00:01:27,810
This can make the drone constantly track the dejected face by calculating the area of the frame and

17
00:01:27,810 --> 00:01:28,770
the face.

18
00:01:28,770 --> 00:01:34,250
We can also determine the occupancy rate if the occupancy rate is too high.

19
00:01:34,260 --> 00:01:37,070
It means that the drone is too close to the face.

20
00:01:37,200 --> 00:01:39,840
So the drone needs to fly back.

21
00:01:39,840 --> 00:01:45,770
If it's too low it means that the drone needs to fly forward and get closer to the face.

22
00:01:46,200 --> 00:01:52,890
Based on this idea we will be creating our codes go to the drone manager and scroll down to the video

23
00:01:52,910 --> 00:01:57,310
JPEG generator where we set the for loop to detect the face.

24
00:01:57,330 --> 00:02:00,590
First we need to calculate the face center X..

25
00:02:00,630 --> 00:02:04,450
This will be X plus half of the width.

26
00:02:04,650 --> 00:02:11,430
We will do the same for calculating the face center Y which will be Y plus half the height.

27
00:02:11,430 --> 00:02:19,170
Let's see in the image the face center will be determined by adding half the with 2 x and half the height

28
00:02:19,170 --> 00:02:20,650
to Y.

29
00:02:20,670 --> 00:02:26,430
Next we want to get the gap between the frame Center and the space center.

30
00:02:26,430 --> 00:02:30,440
Let's set this using div as indifference.

31
00:02:30,470 --> 00:02:38,250
Def X will be set with frame center X which we created previously minus face center X..

32
00:02:38,820 --> 00:02:48,930
This will be the same for height div y will be set to frame center Y minus face center Y then we calculate

33
00:02:48,930 --> 00:02:53,710
the face area which will be with times height.

34
00:02:54,030 --> 00:02:57,110
Next we calculate the face area percentage.

35
00:02:57,420 --> 00:02:59,130
Let's call this percentage.

36
00:02:59,130 --> 00:03:03,230
Percent face which will be face area divided by frame.

37
00:03:03,240 --> 00:03:10,310
Area the drone will be flying based on these numbers to attract the face.

38
00:03:10,350 --> 00:03:13,490
Let's check on the API that we will be using.

39
00:03:13,650 --> 00:03:17,160
We will be using this go x y z speed.

40
00:03:17,160 --> 00:03:23,490
This command is used when you want to fly the drone to determine point x y z under the velocity of a

41
00:03:23,490 --> 00:03:24,740
centimeter per second.

42
00:03:25,970 --> 00:03:34,730
And the description it shows that each value for X Y and Z should be from negative 500 to 500 remembers.

43
00:03:34,740 --> 00:03:43,720
Check the note that says the value for x y and z can not set to my negative 20 to twenty simultaneously.

44
00:03:43,860 --> 00:03:47,680
In this lesson we will be using 30 centimetres.

45
00:03:47,690 --> 00:03:51,660
Now I want to take a moment to explain the x y and z.

46
00:03:51,660 --> 00:03:53,410
Here is an image.

47
00:03:53,570 --> 00:04:01,570
This middle point will be where the drone is that this arrow which is where the X is will mean forward.

48
00:04:01,820 --> 00:04:07,630
The arrow Z means up minus Y means right.

49
00:04:07,690 --> 00:04:13,650
So if you want to fly the drone forward you need to add value to X to fly up.

50
00:04:13,660 --> 00:04:17,910
You will be adding value to Z to fly right.

51
00:04:17,910 --> 00:04:22,790
You will increase the value at minus Y.

52
00:04:22,980 --> 00:04:26,650
Some of you may feel awkward having y on the left side.

53
00:04:26,760 --> 00:04:33,030
There may be changes in this area so you may want to check when there is an update to the SDK or if

54
00:04:33,030 --> 00:04:35,450
there is a far more update on your drone.

55
00:04:36,030 --> 00:04:40,970
If you feel that the drone is not flying correctly you might want to check on the changes.

56
00:04:41,310 --> 00:04:50,870
At this moment which is April 20 19 the value for right will be minus Y for flying down is minus Z.

57
00:04:50,880 --> 00:04:58,920
Flying back will be minus X and flying left is why I think it will be easier to understand once we run

58
00:04:58,920 --> 00:04:59,550
the code.

59
00:04:59,820 --> 00:05:03,520
So let's go back to the script and start making the codes.

60
00:05:03,870 --> 00:05:07,030
Scroll down a bit and set the initial values.

61
00:05:07,170 --> 00:05:17,680
Drone X drone y drone Z and speed x y z will be set to zero and the speed will be self speed.

62
00:05:17,700 --> 00:05:27,090
Now we set and if statement if diff X is smaller than negative 30 the drone y needs to be negative 30

63
00:05:30,590 --> 00:05:33,050
to set the face center in the middle.

64
00:05:33,050 --> 00:05:37,070
We need to fly the drone to the right by 30.

65
00:05:37,070 --> 00:05:41,680
That is the reason we set drone y to negative 30.

66
00:05:41,690 --> 00:05:44,870
You might think why we set this at 30.

67
00:05:44,870 --> 00:05:50,540
If we set it to a smaller number the drone will be constantly moving around every time the face makes

68
00:05:50,540 --> 00:05:54,410
a slight move to have the drone fly smoothly.

69
00:05:54,410 --> 00:05:58,600
30 will be a good number if you prefer a different value.

70
00:05:58,610 --> 00:06:00,350
You can change interior preferred.

71
00:06:00,360 --> 00:06:02,540
No.

72
00:06:02,570 --> 00:06:03,650
Let's go on.

73
00:06:03,650 --> 00:06:10,450
If the diff X is bigger than 30 we need to fly the drone to the left.

74
00:06:10,550 --> 00:06:14,220
So drone y will be set to 30.

75
00:06:14,300 --> 00:06:17,480
Next is to fly down the drone.

76
00:06:17,480 --> 00:06:22,910
If the diff Y is smaller than negative 15 we need to fly the drone down.

77
00:06:22,910 --> 00:06:25,990
So drone Z is set to negative 30.

78
00:06:27,350 --> 00:06:33,820
If diff Y is bigger than 15 the drone Z is set to 30.

79
00:06:33,830 --> 00:06:40,360
Next is the face percentage if percent face is larger than 30 percent.

80
00:06:40,370 --> 00:06:43,980
This means that the drone is too close to the face.

81
00:06:44,090 --> 00:06:47,650
Here we need to set drone X to negative 30.

82
00:06:47,720 --> 00:06:53,240
So the drone can backup if percent face is smaller than 2 percent.

83
00:06:53,240 --> 00:06:56,510
This means that the drone is too far from the face.

84
00:06:56,720 --> 00:07:01,760
Here we need to set drone X to 30 so the drone can come forward.

85
00:07:01,760 --> 00:07:13,100
Lastly we need to send this command self command a string go with drone X drone y drone Z and speed.

86
00:07:13,100 --> 00:07:16,930
If we run this code as it is there will be some problems.

87
00:07:17,210 --> 00:07:22,160
Once we run this code the drone will fly based on the command go.

88
00:07:22,360 --> 00:07:25,030
Let's check the send command method.

89
00:07:25,090 --> 00:07:29,590
Here we have a while loop that is waiting for a response.

90
00:07:29,590 --> 00:07:32,380
The drone may take a few seconds to start flying.

91
00:07:33,130 --> 00:07:39,670
Meanwhile the face detect loop will be processing every millisecond and producing video images while

92
00:07:39,670 --> 00:07:42,240
the send command is waiting for a response.

93
00:07:42,250 --> 00:07:48,070
The video image will be stored and when it goes to the video binary generator the video will show a

94
00:07:48,070 --> 00:07:51,010
delayed video to solve this problem.

95
00:07:51,010 --> 00:07:54,670
We will be setting the CENTCOM in to a thread.

96
00:07:54,700 --> 00:07:58,230
First we need to add an attribute to the initial settings.

97
00:07:58,480 --> 00:08:00,930
So scroll up to the minute it.

98
00:08:00,970 --> 00:08:04,020
Let's set the thread above the send command.

99
00:08:04,150 --> 00:08:07,690
Here we will be using command semaphore.

100
00:08:07,690 --> 00:08:14,440
This will limit the number of command that will process if there is a good man sent while one command

101
00:08:14,440 --> 00:08:21,550
is processing the later command will be on hold or will try the process again later.

102
00:08:21,670 --> 00:08:24,420
We can set the semaphore to one next.

103
00:08:24,550 --> 00:08:26,380
We set the command thread.

104
00:08:26,380 --> 00:08:29,150
Since we haven't set this yet we will set it to none.

105
00:08:30,400 --> 00:08:32,620
Now let's go to the send command.

106
00:08:32,620 --> 00:08:37,540
We don't want to change the name of the method so let's play some underscore in front of the sudden

107
00:08:37,570 --> 00:08:41,600
command then defined said command.

108
00:08:41,710 --> 00:08:47,910
By doing this we can use the original send command as how it is in the news send command.

109
00:08:47,950 --> 00:08:57,050
We set self command and blocking to true next is self command thread and set threading thread.

110
00:08:57,460 --> 00:09:05,140
The target will be the original send command with the underscore arcs will be command and blocking.

111
00:09:05,200 --> 00:09:06,930
Next we want to run this thread.

112
00:09:07,150 --> 00:09:09,930
So let's set self command thread to start

113
00:09:13,670 --> 00:09:15,320
we've added the blocking option.

114
00:09:15,320 --> 00:09:19,220
So we need to add it to the original send command and set this to true.

115
00:09:21,110 --> 00:09:24,250
Let me explain what this blocking option does to the commands.

116
00:09:24,260 --> 00:09:30,740
Once the blocking is set to true while the first command is processing the rest of the following commands

117
00:09:30,920 --> 00:09:32,780
will be on hold.

118
00:09:32,780 --> 00:09:37,880
Once the first command finishes the next command waiting in line will be processed.

119
00:09:37,880 --> 00:09:41,090
This may cause some delays which we don't want.

120
00:09:41,120 --> 00:09:42,730
Rumors streaming videos.

121
00:09:42,800 --> 00:09:49,430
If there is a consecutive command like right left right left we want to ignore the commands which comes

122
00:09:49,430 --> 00:09:52,330
in while the drone is flying right.

123
00:09:52,340 --> 00:10:00,350
This is where we set the is acquired to self command semaphore acquire the blocking will be set to blocking

124
00:10:01,250 --> 00:10:03,170
once the process is unlocked.

125
00:10:03,170 --> 00:10:11,570
We want to set a width statement to context lib exit stack as stack and release the lock.

126
00:10:11,780 --> 00:10:19,730
So we set stack call back with self command semaphore release the following process needs to go under

127
00:10:19,730 --> 00:10:21,770
this width statement.

128
00:10:21,770 --> 00:10:23,350
So let's set it back.

129
00:10:26,330 --> 00:10:33,400
In case the lock doesn't come off we need to set the program to log a warning the status will be not

130
00:10:33,400 --> 00:10:36,430
acquired so we know what happens.

131
00:10:36,550 --> 00:10:39,360
Let's go back and check on what we've made.

132
00:10:39,370 --> 00:10:45,760
We receive a command with Sen command if the commands are a consecutive command such as right.

133
00:10:45,760 --> 00:10:46,090
Right.

134
00:10:46,090 --> 00:10:47,860
Right.

135
00:10:47,890 --> 00:10:52,810
If the blocking option is true this blocking option will also become true.

136
00:10:53,410 --> 00:10:59,830
While the first command right is in process the thread will be locked and the second right command will

137
00:10:59,830 --> 00:11:03,680
be on hold once the first lock is released.

138
00:11:03,730 --> 00:11:09,800
The next command right will process this will go on until all the commands are processed.

139
00:11:12,570 --> 00:11:19,950
If the blocking option is false the first command right will be processed although the second and third

140
00:11:19,950 --> 00:11:26,970
command right will not be acquired and will leave a warning log and won't be processed.

141
00:11:26,970 --> 00:11:28,590
Now go back to the go drone.

142
00:11:28,590 --> 00:11:30,630
X y z speed command.

143
00:11:30,810 --> 00:11:34,950
Here we add in the blocking option and set this to false.

144
00:11:34,950 --> 00:11:40,250
Now we've set the process to ignore commands while the drone is processing the first command.

145
00:11:40,440 --> 00:11:45,720
We can avoid the delayed videos once you've finished creating all these codes.

146
00:11:45,720 --> 00:11:51,120
Try it out and see how your drone works.

147
00:11:51,160 --> 00:11:52,840
I'll be running this program as well.

148
00:11:53,770 --> 00:11:56,140
Let's go to the controller page and take off.

149
00:11:56,980 --> 00:11:59,880
I hit the up button so that the drone doesn't go down.

150
00:11:59,880 --> 00:12:01,310
Landing mode.

151
00:12:01,570 --> 00:12:02,210
Now I hit.

152
00:12:02,210 --> 00:12:05,890
Turn the face detect and track mode to on.

153
00:12:06,130 --> 00:12:12,340
I'm going to go in front of the drone so my face can be detected on the left bottom of the screen you

154
00:12:12,340 --> 00:12:14,130
can see the webcam.

155
00:12:14,260 --> 00:12:21,800
The blue rectangle is around my face which means the drone has found my face as I stand up straight.

156
00:12:21,820 --> 00:12:25,870
You can see the drone is tracking my face and flew up.

157
00:12:25,870 --> 00:12:26,940
Let me go to the right.

158
00:12:26,950 --> 00:12:29,670
So the drone can track me.

159
00:12:29,770 --> 00:12:34,880
Well I think the light was too bright so the color balance changed and the drone can interject my face.

160
00:12:35,110 --> 00:12:36,730
Let me move to the left.

161
00:12:37,030 --> 00:12:43,150
You can see that the drone found my face again and is starting to track if I squat down.

162
00:12:43,150 --> 00:12:45,780
You can see that the drone also comes down.

163
00:12:46,180 --> 00:12:51,640
But if my face is out of the frame you'll see that the face detect drops again.

164
00:12:51,640 --> 00:12:56,770
You might want to try this in a bigger room so that you have plenty of space for the drone can track

165
00:12:56,770 --> 00:12:58,400
you.

166
00:12:58,420 --> 00:13:01,040
I hope your drone tracked you down as well.

167
00:13:01,060 --> 00:13:02,470
I'll see you in the next lesson.