WEBVTT

1
00:00:00.375 --> 00:00:07.090
In this session, we will build the fire detection frontend using Next.js with TanStack Query and polling.  

2
00:00:08.299 --> 00:00:14.264
First, we will refer to the official documentation to install and configure TanStack Query.  

3
00:00:30.697 --> 00:00:37.412
Create a providers.tsx file in the same directory as basic layout.tsx and write the following code.

4
00:00:43.877 --> 00:00:49.591
We will refer to the example in the official documentation to write the `providers` file.  

5
00:01:03.021 --> 00:01:07.901
I accidentally omitted this, but you must include `use client` at the top.  

6
00:01:10.695 --> 00:01:15.283
Wrap the `children` part of the layout file with the provider component. 

7
00:01:19.120 --> 00:01:22.665
Run the project and start writing the code in earnest.  

8
00:01:24.459 --> 00:01:30.048
We will first write the UI structure in JSX and then add detailed functionality.  

9
00:02:29.774 --> 00:02:35.572
Now, I will explain the part where the connected camera list is displayed on the screen.  

10
00:02:36.239 --> 00:02:39.826
our code is a system that detects fires using a webcam.  

11
00:02:41.035 --> 00:02:46.875
In this process, we need to select one of the multiple cameras connected to the computer.  

12
00:02:48.084 --> 00:02:52.255
The implementation of fetching camera information will be done later.  

13
00:02:52.839 --> 00:02:55.550
This select tag displays the list of cameras.  

14
00:02:56.676 --> 00:02:59.929
The user can select the desired camera from the list.  

15
00:03:03.433 --> 00:03:07.353
Create a `selectedDevice` state to store the selected camera.  

16
00:03:13.401 --> 00:03:20.867
Set the `onChange` function so that when a camera is selected from the `select` tag, the value updates accordingly.  

17
00:03:32.128 --> 00:03:34.672
This code uses an array called `devices`.  

18
00:03:35.465 --> 00:03:39.219
`devices` contains information about all available cameras.  

19
00:03:41.221 --> 00:03:45.099
Specify the `MediaDeviceInfo` type to enable type support.  

20
00:03:48.394 --> 00:03:55.443
This array is iterated using the map function,  
And each camera's information is displayed as an option tag.  

21
00:04:00.907 --> 00:04:05.787
Set a `key` for each `option` in the `map` and assign a `value` to each `option`.  

22
00:04:09.207 --> 00:04:12.210
Here, `device.label` represents the camera name.  

23
00:04:14.170 --> 00:04:19.259
If there is no name, it defaults to displaying Camera along with part of its ID.  

24
00:04:31.062 --> 00:04:34.274
Create a `state` to store the status of the `stream`.  

25
00:04:37.235 --> 00:04:41.781
Display the `Start` and `Stop` buttons based on the state of `isStreaming`.  

26
00:04:43.241 --> 00:04:48.162
When `isStreaming` is `true`, display `Stop`, and when `false`, display `Start`.  

27
00:04:48.746 --> 00:04:50.915
I will modify the `onClick` part later.  

28
00:04:51.958 --> 00:04:55.878
I will write the `onClick` functions for `Start` and `Stop`.  

29
00:04:57.005 --> 00:04:59.090
Handle errors using `try-catch`.  

30
00:05:01.968 --> 00:05:06.514
Use `getUserMedia` to fetch the camera stream for a specific `deviceId`.  

31
00:05:09.517 --> 00:05:13.229
`selectedDevice` is the ID of the camera selected by the user.  

32
00:05:14.522 --> 00:05:21.279
In other words, the video stream is retrieved based on the `id` of the camera selected in the `select` dropdown.  

33
00:05:22.196 --> 00:05:26.659
Now, I will explain the key part of the code that starts the webcam.  

34
00:05:27.577 --> 00:05:32.582
First, `videoRef.current` is a reference object pointing to the `<video>` tag.  

35
00:05:33.249 --> 00:05:37.837
This means we can directly manipulate the video element on the screen.  

36
00:05:38.546 --> 00:05:40.631
Declare a videoRef using useRef. 

37
00:05:45.511 --> 00:05:49.766
The `stream` obtained from the webcam is assigned to `videoRef.current.srcObject`.  

38
00:05:53.394 --> 00:05:58.316
This allows the video captured by the webcam to be displayed on the screen.  

39
00:05:59.984 --> 00:06:03.321
Next, `await videoRef.current.play();` is executed.  

40
00:06:04.238 --> 00:06:06.908
This is the code that starts video playback.  

41
00:06:07.533 --> 00:06:09.994
Since the `await` keyword is used,    

42
00:06:10.203 --> 00:06:15.041
It waits until the video actually starts before executing the next line of code.

43
00:06:16.042 --> 00:06:18.169
 `setIsStreaming(true);` is executed.  

44
00:06:18.961 --> 00:06:22.382
This indicates that the webcam is currently running.  

45
00:06:26.219 --> 00:06:28.888
Finally, `setIsPolling(true);` is executed.  

46
00:06:29.639 --> 00:06:34.102
This signals that frames should now be periodically sent to the server.  

47
00:06:35.228 --> 00:06:39.148
Log a message to verify that the webcam has started successfully.  

48
00:06:40.733 --> 00:06:45.154
Now, I will explain the part of the code that stops webcam streaming.  

49
00:06:45.738 --> 00:06:50.326
First, the `stopPredict` function is responsible for stopping the webcam.  

50
00:06:51.327 --> 00:06:56.124
To turn off the webcam, we need to clean up the currently running video stream.  

51
00:06:57.208 --> 00:07:01.087
we check whether `videoRef.current` and `videoRef.current.srcObject` exist.  

52
00:07:01.629 --> 00:07:06.843
This ensures that the video element exists and that the webcam is currently running.  

53
00:07:07.552 --> 00:07:11.556
Next, we use `getTracks()` to get all currently active tracks.  

54
00:07:22.275 --> 00:07:25.695
Webcam video data is managed in units called tracks.  

55
00:07:28.364 --> 00:07:30.116
This code stops all tracks.  

56
00:07:31.200 --> 00:07:35.329
In other words, it prevents the webcam from sending any more video.  

57
00:07:35.955 --> 00:07:38.249
Then, we set `videoRef.current.srcObject` to `null`.  

58
00:07:38.791 --> 00:07:42.003
This removes the webcam stream from the video element.  

59
00:07:42.670 --> 00:07:45.590
As a result, the video disappears from the screen.  

60
00:07:45.840 --> 00:07:49.927
Finally, `setIsStreaming(false);` → Sets the state to indicate that the webcam has stopped.  

61
00:07:50.511 --> 00:07:55.600
Then, `setIsPolling(false);` → Signals that the process of sending data to the server should also stop.  

62
00:07:57.185 --> 00:08:01.647
I will now modify the `onClick` function that was left unchanged earlier.  

63
00:08:02.190 --> 00:08:09.071
Connect the `startPredict` function to the `Start` button and the `stopPredict` function to the `Stop` button.  

64
00:08:10.364 --> 00:08:15.244
Now, I will explain the part that displays the webcam video on the screen.  

65
00:08:16.537 --> 00:08:23.377
First, the `<video>` tag is responsible for displaying the video received from the webcam on the screen.  

66
00:08:24.462 --> 00:08:30.510
Here, `ref` is used to connect `videoRef` to the video element, allowing direct manipulation.  

67
00:08:31.260 --> 00:08:34.263
This enables JavaScript code to control the video.  

68
00:08:35.056 --> 00:08:38.351
The `autoPlay` option makes the video play automatically.  

69
00:08:38.726 --> 00:08:44.065
Once the webcam is connected, the user does not need to press a play button manually.  

70
00:08:44.899 --> 00:08:52.823
The `playsInline` setting ensures that the video does not switch to fullscreen on mobile devices but plays inline instead.  

71
00:08:53.366 --> 00:08:57.495
This helps users watch the video seamlessly within the webpage.  

72
00:08:58.079 --> 00:09:00.706
I will make some simple CSS adjustments.  

73
00:09:17.974 --> 00:09:22.228
Now, I will explain the code that searches for camera devices.  

74
00:09:23.104 --> 00:09:27.650
This code uses `useEffect` to run only once when the page is first loaded.  

75
00:09:35.658 --> 00:09:41.122
The `enumerateDevices` method retrieves all media devices connected to the computer.  

76
00:09:47.628 --> 00:09:53.884
Among them, only video input devices, meaning cameras, are selected through filtering.  

77
00:10:02.184 --> 00:10:06.063
The retrieved camera list is then stored using `setDevices`.  

78
00:10:06.647 --> 00:10:12.820
Additionally, if at least one camera is connected,  
The first camera is selected by default.  

79
00:10:27.084 --> 00:10:35.301
In summary, this code retrieves and stores the list of available cameras,  
And automatically selects the default camera.  

80
00:10:41.724 --> 00:10:44.685
Finally, we call the `getDevices` function.

81
00:10:49.523 --> 00:10:53.444
Now, I will explain the code that captures a video frame.  

82
00:10:53.903 --> 00:10:57.740
This function converts the current webcam screen into an image.  

83
00:11:17.760 --> 00:11:21.097
First, it checks whether the video size is correct.  

84
00:11:22.098 --> 00:11:26.977
If there is no size, capturing is not possible, so the process stops.  

85
00:11:44.412 --> 00:11:50.918
Next, a canvas element is created,  
And its width and height are set according to the video size.  

86
00:12:16.277 --> 00:12:20.156
`getContext('2d')` is used to prepare for drawing 2D graphics.  

87
00:12:39.842 --> 00:12:44.555
Then, `drawImage(videoElement, 0, 0);` is executed,  
Drawing the video frame onto the canvas.  

88
00:12:53.689 --> 00:12:59.862
Finally, `canvas.toBlob()` is used  
To convert the image into a JPEG-format Blob data.  

89
00:13:10.080 --> 00:13:15.419
In summary, this code allows the current video frame to be saved as an image.  

90
00:13:30.434 --> 00:13:35.105
Now, I will explain the code that sends a video frame to the server.  

91
00:13:49.328 --> 00:13:52.623
First, it checks whether the webcam is running.  

92
00:13:53.207 --> 00:14:01.715
If `videoRef.current` does not exist or `isStreaming` is `false`,  
It means the webcam is not running, so the process stops.  

93
00:14:09.515 --> 00:14:15.104
Next, `captureFrame(videoRef.current)` is called,
To convert the current video frame into an image.  

94
00:14:21.610 --> 00:14:26.740
If the conversion fails, it does not send the data to the server and stops.  

95
00:14:30.828 --> 00:14:33.956
Now, prepare to send the image to the server.  

96
00:14:34.290 --> 00:14:38.878
Create a `FormData` object,  
And append the image with the key `'file'`.  

97
00:14:52.016 --> 00:14:55.227
Now, use `fetch` to send the data to the server.  

98
00:14:58.522 --> 00:15:02.318
The request is sent to `"http://localhost:8000/predict_fire"`.  

99
00:15:06.572 --> 00:15:11.160
The request method is `POST`,  
And the request body contains `formData`.  

100
00:15:16.248 --> 00:15:20.127
Finally, convert the server response to JSON and log it. 

101
00:15:32.848 --> 00:15:39.772
In summary, this code captures a video frame,  
And sends it to the server to request fire detection.  

102
00:15:41.565 --> 00:15:47.029
Now, I will explain the code that sends automatic requests using TanStack Query.  

103
00:15:51.367 --> 00:15:54.244
First, `queryKey` is used for caching data. 

104
00:15:56.372 --> 00:16:00.417
Here, the key `"fireDetection"` is used to manage the results.  

105
00:16:01.210 --> 00:16:04.672
`queryFn` is the function that will be executed.  

106
00:16:05.381 --> 00:16:14.223
This executes the `sendFrameToServer` function,  
Which sends a video frame to the server and retrieves the fire detection results.  

107
00:16:15.307 --> 00:16:19.103
`enabled` determines whether the request should be executed. 

108
00:16:19.478 --> 00:16:24.066
It only runs if the webcam is active and data transmission is enabled.  

109
00:16:26.110 --> 00:16:30.823
`refetchInterval` sets the interval for automatically refreshing the data.  

110
00:16:31.991 --> 00:16:35.786
Here, it sends a request to the server every 5 seconds.  

111
00:16:36.245 --> 00:16:41.917
Finally, `retry` determines whether to automatically retry the request if it fails.  

112
00:16:42.418 --> 00:16:44.545
In this case, retries are disabled.  

113
00:16:44.962 --> 00:16:49.174
In summary, this code ensures that when the webcam is running,  

114
00:16:49.174 --> 00:16:52.928
It automatically sends an image to the server every 5 seconds,

115
00:16:52.928 --> 00:16:55.389
And retrieves the fire detection results. 

116
00:16:56.306 --> 00:16:58.517
Check the `data` in the `console.log`.  

117
00:17:08.152 --> 00:17:11.572
To resolve CORS issues when making API requests,

118
00:17:11.572 --> 00:17:15.909
we added the following code to the `main.py` file in FastAPI.  

119
00:17:16.326 --> 00:17:19.997
We used `CORSMiddleware` to allow requests from all domains.  

120
00:17:27.546 --> 00:17:30.424
Requests are allowed from all origins, 

121
00:17:30.424 --> 00:17:33.510
with all HTTP methods and headers permitted.  

122
00:17:34.511 --> 00:17:38.932
This configuration should only be used in a development environment,  

123
00:17:38.932 --> 00:17:41.268
And in a production environment,

124
00:17:41.727 --> 00:17:44.772
it is recommended to apply restrictions as needed. 

125
00:17:45.689 --> 00:17:49.610
Finally, I will correct a few mistakes and wrap things up.  

126
00:17:50.069 --> 00:17:53.197
Add `"use client"` at the top of the `providers` file.  

127
00:18:00.287 --> 00:18:02.498
I will update the API request URL.  

128
00:18:07.002 --> 00:18:09.963
I will check the functionality in the browser.  

129
00:18:15.052 --> 00:18:19.598
If a fire is detected, `"fire detection"` is displayed in the `message`.  

130
00:18:23.852 --> 00:18:30.025
Next time, we will implement a simple modal and alert notification when a fire is detected.