1
00:00:00,150 --> 00:00:05,070
In this lesson we will be streaming from the drone camera onto the webcam.

2
00:00:05,070 --> 00:00:10,520
This wouldn't be an actual streaming but we will be loading the image from the camera.

3
00:00:10,530 --> 00:00:13,020
Now let's see how we will do this.

4
00:00:13,110 --> 00:00:16,120
The drone camera will be filming a video.

5
00:00:16,320 --> 00:00:22,770
The binary will be sent using UDP through the 1 1 1 1 port to the computer.

6
00:00:22,770 --> 00:00:26,160
The data will be processed using f f MPEG.

7
00:00:26,160 --> 00:00:28,880
Here we have the f f MPEG Web site.

8
00:00:29,040 --> 00:00:36,450
For those of you who are using a Mac you've already installed f f MPEG when you install the open C V.

9
00:00:36,720 --> 00:00:41,970
For those of you using windows we've installed the f f MPEG individually.

10
00:00:42,270 --> 00:00:50,550
If you're not sure installing go to the command line and type in f f MPEG to see if it runs we will

11
00:00:50,550 --> 00:00:57,810
be using the f f MPEG command to run the video through pipe 0 so the process will start to decode the

12
00:00:57,810 --> 00:01:04,710
video so it can run on the computer after the process finishes the video will be sent through pipe 1

13
00:01:05,670 --> 00:01:09,650
and the image will be showing in the webcam.

14
00:01:09,760 --> 00:01:12,670
That's the overview of what we'll be creating.

15
00:01:12,670 --> 00:01:15,720
Let's dive right into the script to make it work.

16
00:01:16,460 --> 00:01:19,890
First let's start opening up the drone manager P Y.

17
00:01:20,330 --> 00:01:24,950
The file will be located under models which is under drone apps.

18
00:01:25,460 --> 00:01:32,270
Since we're at the top of the script we'll import the packages that we will be using later import O

19
00:01:32,300 --> 00:01:34,850
S and said process.

20
00:01:34,850 --> 00:01:43,760
Make sure you put it in alphabetical orders then import third party packages such as CV to a CV and

21
00:01:43,760 --> 00:01:46,800
not be as MP.

22
00:01:46,820 --> 00:01:55,580
Now let's go down a bit and declare the frame size globally frame X which will be the width is 960 for

23
00:01:55,580 --> 00:02:04,030
the Tello camera frame Y which is the height will be 720 if we set the frame size too big.

24
00:02:04,070 --> 00:02:06,380
The process time will be longer.

25
00:02:06,380 --> 00:02:10,210
This is something we want to avoid when we start face recognition.

26
00:02:10,250 --> 00:02:15,320
So let's set the size to third by dividing the numbers by three.

27
00:02:15,330 --> 00:02:20,180
Also we need to set the frame area which will be frame x times frame y

28
00:02:22,680 --> 00:02:30,980
next set frame size which will be frame size times 3 when the f f MPEG is in process.

29
00:02:31,050 --> 00:02:38,610
It will receive not only the first frame but also many other data such as a r b value to process all

30
00:02:38,610 --> 00:02:39,330
these data.

31
00:02:39,330 --> 00:02:44,310
We will need about three times the frame area size later on.

32
00:02:44,310 --> 00:02:46,470
We will need to use the frame position.

33
00:02:46,770 --> 00:02:51,450
So let's set the frame centre X to frame x divided by two

34
00:02:54,210 --> 00:03:00,220
also set frame centre y to frame y divided by two.

35
00:03:00,230 --> 00:03:11,070
Next we need to set how we will be using the pipes set command f of MPEG to f string FFL pen then followed

36
00:03:11,070 --> 00:03:17,910
by H W a CCL where we choose to process on hard or soft.

37
00:03:17,920 --> 00:03:20,920
This will be to auto next.

38
00:03:21,150 --> 00:03:31,860
H W ACL device will be open C L and output will be pipe 0 P I underscore F.M. t will be where it to

39
00:03:31,860 --> 00:03:41,460
determine the colour which we will set to BGR 24 size will be set to frame X and frame y.

40
00:03:41,500 --> 00:03:45,820
Lastly hyphen F will be where we send the raw video.

41
00:03:45,880 --> 00:03:52,600
This will be set to pipe one I've created quite a long code so let's divide these into different rows

42
00:03:52,610 --> 00:03:55,750
so you can see it better when you separate the row.

43
00:03:55,750 --> 00:03:59,030
You need to use the parentheses.

44
00:03:59,160 --> 00:04:04,090
Now we will go onto the drone manager in it before we run the process.

45
00:04:04,110 --> 00:04:10,620
We need to set self proc to so process P open and run the command.

46
00:04:10,680 --> 00:04:15,090
F F MPEG to split with the space and return as a list

47
00:04:18,630 --> 00:04:25,020
a study in will be sub process pipe and a C.D. out will also be some process pipe

48
00:04:27,660 --> 00:04:34,160
self proc a G in will be set to self proc SCADA and also self proc.

49
00:04:34,170 --> 00:04:43,570
A study out will be set to self proc a study out the video part will be 1 1 1 1 1.

50
00:04:43,590 --> 00:04:46,410
Next we need to set a thread.

51
00:04:46,410 --> 00:04:56,850
Self received video thread set to threading thread target will be self received video currently received

52
00:04:56,850 --> 00:04:58,520
video doesn't exist.

53
00:04:58,630 --> 00:05:03,990
So we need to create it later args will be set to self stop event.

54
00:05:07,930 --> 00:05:16,880
Self proc AC in self host IP and self video port once the args are set.

55
00:05:16,890 --> 00:05:22,610
We need to set receive video thread start to start the process.

56
00:05:22,650 --> 00:05:28,410
Next we need to create the receive video scroll all the way to the very bottom of the script.

57
00:05:28,680 --> 00:05:38,370
Here will define receive video followed by self stop event by bean host IP and video port.

58
00:05:38,380 --> 00:05:43,660
Now we need to set how the binary data from the drone video will be received.

59
00:05:43,660 --> 00:05:50,870
First we start with a width stock at we use self sockets so it can be used in other methods.

60
00:05:50,870 --> 00:05:54,320
But in this case we won't be using it in other methods.

61
00:05:54,320 --> 00:06:04,340
So we will use the w statement with socket socket and set socket a f I any t to use the network and

62
00:06:04,340 --> 00:06:05,710
we'll use UDP.

63
00:06:05,810 --> 00:06:10,330
So we set socket socket diagram.

64
00:06:10,440 --> 00:06:14,460
This is as SOC video next.

65
00:06:14,490 --> 00:06:25,230
Let's set the socket video using set SOC O.P. t set socket aswell socket and socket as O reuse address

66
00:06:25,350 --> 00:06:28,590
and set one by setting this code.

67
00:06:28,590 --> 00:06:32,880
Once the socket video is released the address can be reused.

68
00:06:32,880 --> 00:06:40,270
So if we reboot the system the address can be reused let's go on and set the time out time.

69
00:06:40,420 --> 00:06:43,630
Here we'll set it to point five seconds.

70
00:06:43,630 --> 00:06:51,580
Next we bind the SOC video host IP and video port this should be a tuple type so don't forget to set

71
00:06:51,580 --> 00:06:54,460
it between the parentheses.

72
00:06:54,460 --> 00:06:58,110
We also need to set the byte every two 2048.

73
00:06:58,180 --> 00:07:05,710
There wouldn't be any problem if we set it to 10 24 but in the drawn SDK it was set to 2048 so the system

74
00:07:05,710 --> 00:07:08,950
can receive the large video while not.

75
00:07:08,950 --> 00:07:13,570
Will it be a loop that goes on until the stop event is set.

76
00:07:14,020 --> 00:07:18,610
Next try size and address will be SOC video receive from.

77
00:07:18,640 --> 00:07:21,610
Into and set data to get the byte airy

78
00:07:27,720 --> 00:07:32,880
both the size and address will be set into each size and address.

79
00:07:32,880 --> 00:07:40,290
If we won't be using this we can set it to underscore if you wish to log the data you can leave it as

80
00:07:40,290 --> 00:07:41,640
it is.

81
00:07:41,640 --> 00:07:45,900
If the process takes over point five seconds we want to receive a warning.

82
00:07:45,960 --> 00:07:47,870
So let's set socket timeout.

83
00:07:47,880 --> 00:07:52,380
As exception and leave a logger so we get a warning.

84
00:07:52,430 --> 00:07:54,340
Action will be received video

85
00:07:57,230 --> 00:08:04,700
E X will be X after logging we will wait for another point five seconds to retry.

86
00:08:04,700 --> 00:08:05,390
Here we set.

87
00:08:05,390 --> 00:08:06,130
Continue.

88
00:08:06,170 --> 00:08:12,470
So the while loop goes on another exception will be when there is a socket error.

89
00:08:12,580 --> 00:08:18,200
If there is an error occurring then we will break the loop before we break the loop.

90
00:08:18,200 --> 00:08:23,630
We want to log the error with action receive video and E X as the X

91
00:08:27,060 --> 00:08:32,820
we just created the part where we see the video sent from the drone through the bind socket.

92
00:08:32,850 --> 00:08:36,120
The data will be sent into variable data.

93
00:08:36,120 --> 00:08:41,290
Next we want to connect the data to the F F MPEG pipe in set.

94
00:08:41,300 --> 00:08:53,510
Try then set pipe and to write the data with its size we set the data size to 2048 but that won't be

95
00:08:53,510 --> 00:08:55,170
the size of the actual data.

96
00:08:55,280 --> 00:09:03,500
It may be a thousand or even nine whatever size the received data is will be set to the data size.

97
00:09:03,610 --> 00:09:06,080
Next we need to flush the data into the pipe.

98
00:09:06,110 --> 00:09:09,790
0 here we also want to catch errors.

99
00:09:09,820 --> 00:09:15,950
So let's set an exception and leave a logger so we know what happened in case of an error.

100
00:09:15,970 --> 00:09:19,710
We'll have to break the loop.

101
00:09:20,080 --> 00:09:24,480
We just finished the part where the drone video has reached the pipe.

102
00:09:24,580 --> 00:09:30,050
Once we've got to this point let's test the script set a logger here.

103
00:09:30,400 --> 00:09:36,530
I'll copy the log error code and change it to a log info and set to show the data.

104
00:09:36,610 --> 00:09:39,580
Next we want to set the stop function.

105
00:09:39,580 --> 00:09:45,410
Once we stop the process we also want to stop the F of epic process as well.

106
00:09:45,640 --> 00:09:53,250
The video received will stop once the thread stops so that's stop here at the bottom of the stop function.

107
00:09:53,380 --> 00:10:02,830
We can set o s kill to self proc PD to 9 if you are familiar with Linux you might find it easier to

108
00:10:02,830 --> 00:10:03,460
understand.

109
00:10:03,550 --> 00:10:04,830
I'll show you in the terminal.

110
00:10:04,830 --> 00:10:11,350
We command P S with the system that is running you'll see the process information.

111
00:10:11,350 --> 00:10:19,720
These numbers will be the PD you can command kill 9 followed by the certain PD to stop the process.

112
00:10:19,720 --> 00:10:23,920
This code can kill the process as we just saw in the terminal.

113
00:10:24,130 --> 00:10:33,060
For those of you who are using Windows would have to import signal then set OS kill self proc PD signal

114
00:10:33,070 --> 00:10:36,910
control C event to kill the process.

115
00:10:37,450 --> 00:10:43,960
In my case I'm using a Mac so I'll comment out on these codes once you finish all the codes.

116
00:10:43,960 --> 00:10:47,290
Let's run the script and see if we receive the data.

117
00:10:47,830 --> 00:10:53,860
For those of you who are using a Windows device once you connect your drone you need to turn off your

118
00:10:53,860 --> 00:10:59,000
firewall to receive Pt. 1 1 1 1 1.

119
00:10:59,290 --> 00:11:02,970
Remember to disable the firewall settings.

120
00:11:03,100 --> 00:11:05,540
Now I've just started running the script.

121
00:11:05,590 --> 00:11:07,210
Let's go to the controller page.

122
00:11:09,360 --> 00:11:11,280
Now press takeoff.

123
00:11:11,310 --> 00:11:15,540
Once you press the take off button you'll be seeing a lot of log data.

124
00:11:15,540 --> 00:11:20,940
This means we've succeeded in receiving the data from the drone.

125
00:11:21,250 --> 00:11:25,190
We'll be processing on these binary data in our next lesson.

126
00:11:25,210 --> 00:11:28,460
Now let's press the land button to land the drone.

127
00:11:28,600 --> 00:11:32,150
Make sure that the log has also stopped.

128
00:11:32,260 --> 00:11:38,410
Now I don't want all these log data to take up the space so let's comment out on the logger info here

129
00:11:38,950 --> 00:11:40,840
or you can also delete the row.

130
00:11:40,870 --> 00:11:42,170
I'll see you in our next lesson.