﻿1
00:00:01,330 --> 00:00:04,200
‫Welcome back after a long lecture

2
00:00:04,200 --> 00:00:06,470
‫where we searched for tour documents

3
00:00:06,470 --> 00:00:08,170
‫within a certain distance

4
00:00:08,170 --> 00:00:11,930
‫from a certain point using geospatial queries.

5
00:00:11,930 --> 00:00:15,840
‫Now in this lecture, let's use geospatial aggregation

6
00:00:15,840 --> 00:00:17,580
‫in order to calculate distances

7
00:00:17,580 --> 00:00:20,073
‫to all the tours from a certain point.

8
00:00:21,970 --> 00:00:24,320
‫So just like before let's actually start

9
00:00:24,320 --> 00:00:26,510
‫by defining the route so that we know

10
00:00:26,510 --> 00:00:28,610
‫which data we're going to be working with.

11
00:00:31,750 --> 00:00:36,150
‫So router.route;

12
00:00:36,150 --> 00:00:40,760
‫at this time I'm gonna call it simply distances,

13
00:00:40,760 --> 00:00:43,130
‫and then the data that we need

14
00:00:43,130 --> 00:00:45,050
‫is the latitude and the longitude

15
00:00:45,050 --> 00:00:47,193
‫of the point where the user currently is,

16
00:00:48,090 --> 00:00:51,103
‫so in our previous example that was LA,

17
00:00:52,840 --> 00:00:54,310
‫and then let's also allow the user

18
00:00:54,310 --> 00:00:57,273
‫again to specify the unit.

19
00:00:59,370 --> 00:01:02,150
‫Then here, the route handler function.

20
00:01:02,150 --> 00:01:03,060
‫Now this time here,

21
00:01:03,060 --> 00:01:05,850
‫we do not need the distance parameter,

22
00:01:05,850 --> 00:01:07,250
‫as we had it right here,

23
00:01:07,250 --> 00:01:10,600
‫because we're not gonna be searching for a certain radius.

24
00:01:10,600 --> 00:01:12,620
‫We're really gonna calculate the distance

25
00:01:12,620 --> 00:01:14,880
‫from a certain point to all the tours

26
00:01:14,880 --> 00:01:16,683
‫that we have in our collection.

27
00:01:17,530 --> 00:01:20,483
‫So the handler is at tourController,

28
00:01:21,880 --> 00:01:24,077
‫and it's gonna be called getDistances.

29
00:01:28,710 --> 00:01:32,033
‫We don't have that yet, and so let's create it.

30
00:01:37,610 --> 00:01:42,210
‫CatchAsync, and then of course

31
00:01:42,210 --> 00:01:45,170
‫mark the function as async as well

32
00:01:45,170 --> 00:01:47,620
‫because we already know that we're gonna use

33
00:01:47,620 --> 00:01:50,210
‫the aggregation pipeline, and so by the time

34
00:01:50,210 --> 00:01:52,393
‫we're gonna be using a wait.

35
00:01:57,771 --> 00:02:02,070
‫The beginning of this function is actually quite similar

36
00:02:02,070 --> 00:02:05,470
‫to the getToursWithin one, and so let's go ahead

37
00:02:05,470 --> 00:02:07,223
‫and just copy all of this code.

38
00:02:08,730 --> 00:02:11,830
‫We have some similar units, then we also need to get

39
00:02:11,830 --> 00:02:13,450
‫the latitude and longitude,

40
00:02:13,450 --> 00:02:15,520
‫and we also need to create this error

41
00:02:15,520 --> 00:02:18,583
‫in case there is no latitude or no longitude.

42
00:02:21,980 --> 00:02:23,890
‫This one here does not apply,

43
00:02:23,890 --> 00:02:25,833
‫and also we don't have the distance.

44
00:02:28,526 --> 00:02:30,713
‫So let's now do the actual calculation.

45
00:02:31,800 --> 00:02:34,660
‫Just like before in order to do calculations

46
00:02:34,660 --> 00:02:37,730
‫we always use the aggregation pipeline.

47
00:02:37,730 --> 00:02:40,513
‫And remember, that is called on the model itself.

48
00:02:41,520 --> 00:02:43,923
‫So Tour.aggregate.

49
00:02:45,800 --> 00:02:48,830
‫Then let's await that, and save it

50
00:02:48,830 --> 00:02:50,723
‫into the distances variable.

51
00:02:55,349 --> 00:02:58,020
‫So then here, remember, we passed in an array

52
00:02:58,020 --> 00:03:00,803
‫with all the stages of the aggregation pipeline

53
00:03:00,803 --> 00:03:02,700
‫that we want to define.

54
00:03:02,700 --> 00:03:04,660
‫Now for geospatial aggregation,

55
00:03:04,660 --> 00:03:07,550
‫there's actually only one single stage,

56
00:03:07,550 --> 00:03:09,967
‫and that's called geoNear, so this one.

57
00:03:16,380 --> 00:03:18,518
‫Again, this is the only geospatial

58
00:03:18,518 --> 00:03:21,780
‫aggregation pipeline stage that actually exists.

59
00:03:21,780 --> 00:03:26,530
‫This one always needs to be the first one in the pipeline.

60
00:03:26,530 --> 00:03:28,840
‫So keep that in mind that geoNear

61
00:03:28,840 --> 00:03:31,173
‫always needs to be the first stage.

62
00:03:32,620 --> 00:03:35,700
‫Something else that's also very important to note

63
00:03:35,700 --> 00:03:38,370
‫about geoNear is that it requires

64
00:03:38,370 --> 00:03:40,430
‫that at least one of our fields

65
00:03:40,430 --> 00:03:42,713
‫contains a geospatial index.

66
00:03:43,930 --> 00:03:46,440
‫Actually we already did that before,

67
00:03:46,440 --> 00:03:48,965
‫so let's again take a look.

68
00:03:48,965 --> 00:03:51,290
‫Our start location already has

69
00:03:51,290 --> 00:03:54,895
‫this 2dsphere geospatial index on it.

70
00:03:54,895 --> 00:03:57,290
‫Since we're using this startLocation

71
00:03:57,290 --> 00:03:59,390
‫in order to calculate the distances,

72
00:03:59,390 --> 00:04:01,593
‫well, that's then perfect.

73
00:04:03,050 --> 00:04:06,138
‫If there's only one field with a geospatial index

74
00:04:06,138 --> 00:04:10,120
‫then this geoNear stage here will automatically use

75
00:04:10,120 --> 00:04:13,490
‫that index in order to perform the calculation.

76
00:04:13,490 --> 00:04:16,570
‫But if you have multiple fields with geospatial indexes

77
00:04:16,570 --> 00:04:18,880
‫then you need to use the keys parameter

78
00:04:18,880 --> 00:04:20,440
‫in order to define the field

79
00:04:20,440 --> 00:04:22,623
‫that you want to use for calculations.

80
00:04:24,429 --> 00:04:26,120
‫So keep that in mind, but again,

81
00:04:26,120 --> 00:04:27,960
‫in this case we only have one field,

82
00:04:27,960 --> 00:04:30,850
‫and so automatically that startLocation field

83
00:04:30,850 --> 00:04:33,740
‫is going to be used for doing these calculations.

84
00:04:33,740 --> 00:04:37,230
‫So, what do we need to pass into geoNear?

85
00:04:37,230 --> 00:04:41,396
‫Well, first we need to specify the near property,

86
00:04:41,396 --> 00:04:45,800
‫and near is the point from which to calculate the distances.

87
00:04:45,800 --> 00:04:49,030
‫So all the distances will be calculated from this point

88
00:04:49,030 --> 00:04:52,410
‫that we define here, and then all the start locations.

89
00:04:52,410 --> 00:04:54,804
‫So this near point here is of course

90
00:04:54,804 --> 00:04:57,602
‫the point that we pass into this function

91
00:04:57,602 --> 00:04:59,743
‫with this latitude and longitude.

92
00:05:01,496 --> 00:05:05,773
‫Now we need to specify this point here as geojson,

93
00:05:06,890 --> 00:05:09,180
‫so that's just like we did it before,

94
00:05:09,180 --> 00:05:12,153
‫where we need to specify the type as Point,

95
00:05:14,517 --> 00:05:17,647
‫and then specify the coordinates property.

96
00:05:20,320 --> 00:05:23,423
‫And as always the first coordinate here is the longitude,

97
00:05:25,640 --> 00:05:28,530
‫and then the second one, the latitude.

98
00:05:28,530 --> 00:05:31,520
‫And let's multiply both of them by one,

99
00:05:31,520 --> 00:05:34,053
‫simply to convert it to numbers.

100
00:05:36,240 --> 00:05:40,060
‫So this is the first mandatory field, near,

101
00:05:40,060 --> 00:05:43,563
‫and the second one is the distance field property.

102
00:05:46,160 --> 00:05:48,870
‫So, distanceField, and so this is the name

103
00:05:48,870 --> 00:05:51,090
‫of the field that will be created

104
00:05:51,090 --> 00:05:54,270
‫and where all the calculated distances will be stored.

105
00:05:54,270 --> 00:05:57,653
‫So let's simply call this one distance.

106
00:05:59,710 --> 00:06:01,660
‫Actually, that's it.

107
00:06:01,660 --> 00:06:03,770
‫That's all the fields that are mandatory

108
00:06:03,770 --> 00:06:06,180
‫in this geoNear stage.

109
00:06:06,180 --> 00:06:08,560
‫And of course, we can add other stages here,

110
00:06:08,560 --> 00:06:10,740
‫and we're actually going to do that a bit later,

111
00:06:10,740 --> 00:06:12,570
‫but for now all I want to do

112
00:06:12,570 --> 00:06:15,573
‫is to really see the results of this working.

113
00:06:17,670 --> 00:06:22,410
‫Let's again copy this a result here,

114
00:06:22,410 --> 00:06:27,410
‫thus sending these results, and here,

115
00:06:27,600 --> 00:06:29,503
‫let's then send the distances,

116
00:06:30,810 --> 00:06:32,410
‫and also this one we don't need.

117
00:06:35,524 --> 00:06:37,860
‫So, we're ready to start.

118
00:06:37,860 --> 00:06:40,680
‫Keep in mind that at this point we didn't use the unit,

119
00:06:40,680 --> 00:06:42,370
‫but don't worry about that.

120
00:06:42,370 --> 00:06:44,580
‫We're gonna do that in a second,

121
00:06:44,580 --> 00:06:47,723
‫but again, first I really want to see this working.

122
00:06:50,320 --> 00:06:52,623
‫Remember that the route is now distances,

123
00:06:55,180 --> 00:06:57,190
‫so let's just copy this one here.

124
00:06:57,190 --> 00:06:58,763
‫Actually, I will save it also,

125
00:06:59,790 --> 00:07:01,223
‫so into the tours.

126
00:07:03,860 --> 00:07:07,710
‫Let's say get tours within radius.

127
00:07:16,540 --> 00:07:21,540
‫This here is called distances,

128
00:07:22,000 --> 00:07:26,290
‫and we do not have this and also not this.

129
00:07:26,290 --> 00:07:29,523
‫So just the coordinates, and then again the unit.

130
00:07:31,040 --> 00:07:35,740
‫Let's take a look, and we will now get this error.

131
00:07:35,740 --> 00:07:38,100
‫Remember how we said that geoNear

132
00:07:38,100 --> 00:07:41,750
‫always needs to be the first stage in a pipeline,

133
00:07:41,750 --> 00:07:43,870
‫but if you now take a look at the code

134
00:07:43,870 --> 00:07:48,510
‫you might think that actually our geoNear stage

135
00:07:48,510 --> 00:07:51,690
‫is currently the first stage of our pipeline.

136
00:07:51,690 --> 00:07:55,290
‫Because right here, it actually looks like it is, right?

137
00:07:55,290 --> 00:07:58,530
‫There's nothing before this, and so why do we get this error

138
00:07:58,530 --> 00:08:02,134
‫that geoNear is not the first stage in the pipeline?

139
00:08:02,134 --> 00:08:06,010
‫Actually it took me a bit of time to figure this out

140
00:08:06,010 --> 00:08:08,730
‫because this has something to do with a piece of code

141
00:08:08,730 --> 00:08:10,623
‫that we wrote a long time ago.

142
00:08:12,050 --> 00:08:14,240
‫That's here in the tour model,

143
00:08:14,240 --> 00:08:16,623
‫and if we go down here, I think.

144
00:08:19,480 --> 00:08:22,440
‫Right here, we have this aggregation middleware,

145
00:08:22,440 --> 00:08:26,220
‫and remember that what this did is to actually always add

146
00:08:26,220 --> 00:08:29,840
‫this match stage here before all the other stages,

147
00:08:29,840 --> 00:08:32,050
‫and actually we have this console.log here

148
00:08:32,050 --> 00:08:34,700
‫and so indeed you can actually see

149
00:08:34,700 --> 00:08:36,593
‫the entire pipeline down here.

150
00:08:37,790 --> 00:08:40,060
‫And so you see that we first have the match,

151
00:08:40,060 --> 00:08:42,130
‫and then the geoNear phase here,

152
00:08:42,130 --> 00:08:45,230
‫actually only as the second stage.

153
00:08:45,230 --> 00:08:48,670
‫So it actually makes sense that we get that error.

154
00:08:48,670 --> 00:08:52,120
‫Now we could go ahead and change this middleware here

155
00:08:52,120 --> 00:08:55,630
‫and say that if geoNear is the first operator

156
00:08:55,630 --> 00:08:59,873
‫in the pipeline, then simply do not do this here.

157
00:08:59,873 --> 00:09:03,530
‫But that's a bit too much work for this use case,

158
00:09:03,530 --> 00:09:06,853
‫so all I'm gonna do is to get rid of this middleware.

159
00:09:08,448 --> 00:09:11,833
‫So give this a save, and now let's try it again.

160
00:09:13,929 --> 00:09:16,420
‫Now we get our tours, and now it should have

161
00:09:16,420 --> 00:09:18,633
‫that distance field on them.

162
00:09:19,520 --> 00:09:24,230
‫So let's search for that, and indeed here it goes.

163
00:09:24,230 --> 00:09:27,588
‫So distance, and then this huge number here.

164
00:09:27,588 --> 00:09:29,740
‫It is this big number,

165
00:09:29,740 --> 00:09:32,490
‫because actually it's calculated in meters,

166
00:09:32,490 --> 00:09:35,270
‫so this result comes in meters,

167
00:09:35,270 --> 00:09:38,683
‫so let's first of all convert this one to kilometers.

168
00:09:39,560 --> 00:09:42,630
‫Later on we will then also convert it to miles,

169
00:09:42,630 --> 00:09:46,120
‫because remember we specified the unit to miles,

170
00:09:46,120 --> 00:09:47,860
‫but for now the easiest solution

171
00:09:47,860 --> 00:09:49,960
‫is to actually convert it to kilometers,

172
00:09:49,960 --> 00:09:51,640
‫because all we have to do for that

173
00:09:51,640 --> 00:09:54,920
‫is to just divide it by 1000.

174
00:09:54,920 --> 00:09:56,250
‫And then also what I want to do

175
00:09:56,250 --> 00:09:58,950
‫is to only really get the distances,

176
00:09:58,950 --> 00:10:00,530
‫and the name of the tours.

177
00:10:00,530 --> 00:10:04,170
‫So get rid of all the other clutter that we have here

178
00:10:04,170 --> 00:10:07,133
‫and really only focus on the distances themselves.

179
00:10:08,610 --> 00:10:11,160
‫For that, as you might remember,

180
00:10:11,160 --> 00:10:14,350
‫we can use the project stage.

181
00:10:14,350 --> 00:10:17,163
‫So let's add that here as the second stage.

182
00:10:20,160 --> 00:10:24,470
‫So project, and then basically the names

183
00:10:24,470 --> 00:10:26,373
‫of the fields that we want to keep.

184
00:10:27,230 --> 00:10:31,003
‫So that's the distance, and so we set that one to one,

185
00:10:32,100 --> 00:10:35,757
‫saying that we want to keep it, and then also the name

186
00:10:35,757 --> 00:10:39,653
‫so that we actually know what tour we're talking about.

187
00:10:40,990 --> 00:10:43,800
‫With that we get rid of all the other data,

188
00:10:43,800 --> 00:10:47,220
‫and now let's basically divide the distance by 1000

189
00:10:47,220 --> 00:10:50,320
‫in order to convert these meters to kilometers.

190
00:10:50,320 --> 00:10:52,590
‫Actually, it's very easy to do that,

191
00:10:52,590 --> 00:10:56,249
‫because in a geoNear stage we can actually specify

192
00:10:56,249 --> 00:10:59,543
‫the distance multiplier property.

193
00:11:00,410 --> 00:11:05,410
‫So distanceMultiplier, and so here we can specify a number

194
00:11:07,470 --> 00:11:10,790
‫which is then going to be multiplied with all the distances.

195
00:11:10,790 --> 00:11:15,790
‫Here we specify 0.001, and so that is exactly the same

196
00:11:16,080 --> 00:11:17,763
‫as dividing by 1000.

197
00:11:19,860 --> 00:11:21,763
‫So let's test our result here now.

198
00:11:23,210 --> 00:11:25,760
‫And that calculation apparently takes some time,

199
00:11:25,760 --> 00:11:26,983
‫but now here we go.

200
00:11:27,820 --> 00:11:32,050
‫So now you get this nice result here in kilometers.

201
00:11:32,050 --> 00:11:35,200
‫As you see the Sports Lover is the closest tour

202
00:11:35,200 --> 00:11:37,920
‫to the location in Los Angeles that we marked.

203
00:11:37,920 --> 00:11:40,220
‫So it's only 64 kilometers away,

204
00:11:40,220 --> 00:11:42,430
‫which should be something like 40 miles.

205
00:11:42,430 --> 00:11:45,380
‫But again, we're going to do that conversion in a second.

206
00:11:45,380 --> 00:11:48,487
‫For now, I just want to go back to that map and compass

207
00:11:48,487 --> 00:11:50,863
‫and see if this actually makes sense.

208
00:11:53,530 --> 00:11:56,800
‫So we're still here,

209
00:11:56,800 --> 00:11:59,513
‫and we still have our start locations map.

210
00:12:01,100 --> 00:12:01,933
‫Now the problem here

211
00:12:01,933 --> 00:12:04,980
‫is that we actually cannot really click

212
00:12:04,980 --> 00:12:07,433
‫on any of these points and see what they are.

213
00:12:08,960 --> 00:12:11,660
‫But let's just draw a quick circle here again

214
00:12:12,670 --> 00:12:15,000
‫just to see which are the closest tours,

215
00:12:15,000 --> 00:12:17,770
‫and if they match the ones in our output.

216
00:12:17,770 --> 00:12:19,753
‫So it's kind of here, I believe,

217
00:12:21,300 --> 00:12:24,303
‫and so let's include these five tours here.

218
00:12:27,070 --> 00:12:31,400
‫So their names are The Park Camper, Snow Adventurer,

219
00:12:31,400 --> 00:12:34,783
‫Wine Taster, Sports Lover and Star Gazer,

220
00:12:35,640 --> 00:12:37,893
‫and so now when we come here,

221
00:12:39,450 --> 00:12:42,150
‫these ones are actually the first five ones.

222
00:12:42,150 --> 00:12:43,970
‫Sports Lover, Park Camper, Wine Taster,

223
00:12:43,970 --> 00:12:46,700
‫Star Gazer and Snow Adventurer.

224
00:12:46,700 --> 00:12:49,490
‫So that one that's really close is the Sports Lover,

225
00:12:49,490 --> 00:12:51,310
‫and then the next one is the Park Camper

226
00:12:51,310 --> 00:12:52,763
‫and the Wine Taster.

227
00:12:58,079 --> 00:13:00,490
‫This one here is going to be the Park Camper,

228
00:13:00,490 --> 00:13:02,870
‫which I believe starts in Las Vegas,

229
00:13:02,870 --> 00:13:04,300
‫so that makes sense,

230
00:13:04,300 --> 00:13:06,473
‫and then a third one is here,

231
00:13:06,473 --> 00:13:09,780
‫The Wine Taster close to San Francisco.

232
00:13:09,780 --> 00:13:13,160
‫So that distance of 800 kilometers I think,

233
00:13:13,160 --> 00:13:14,113
‫or what was that?

234
00:13:15,290 --> 00:13:19,060
‫Yeah, 600 kilometers, that actually makes kind of sense.

235
00:13:19,060 --> 00:13:21,593
‫So, something close to 400 miles here.

236
00:13:22,910 --> 00:13:25,070
‫And speaking of miles, let's actually do

237
00:13:25,070 --> 00:13:26,513
‫that conversion right now.

238
00:13:28,030 --> 00:13:31,570
‫Let's do something similar to what we did before,

239
00:13:31,570 --> 00:13:33,523
‫so testing for the unit.

240
00:13:34,570 --> 00:13:36,943
‫Let's create a multiplier variable,

241
00:13:39,320 --> 00:13:41,713
‫again a ternary operator here,

242
00:13:45,030 --> 00:13:49,070
‫so if it's miles then what should our multiplier be?

243
00:13:49,070 --> 00:13:50,630
‫Well, let's actually very simply

244
00:13:50,630 --> 00:13:53,703
‫Google what one meter is in miles.

245
00:13:58,320 --> 00:14:03,250
‫One meter to miles, and Google usually gives us

246
00:14:03,250 --> 00:14:08,210
‫a pretty nice response, and so indeed, that it is.

247
00:14:08,210 --> 00:14:11,750
‫So if this is one meter, then all we need to do

248
00:14:11,750 --> 00:14:14,660
‫is to really multiply our result in meters

249
00:14:14,660 --> 00:14:15,863
‫with this number.

250
00:14:16,800 --> 00:14:20,340
‫So let's copy it here, and go back,

251
00:14:20,340 --> 00:14:22,500
‫and so this should be our multiplier

252
00:14:22,500 --> 00:14:26,690
‫in case the unit is meters, or actually in case it's miles.

253
00:14:26,690 --> 00:14:29,030
‫And in case it's meters, well then

254
00:14:29,030 --> 00:14:32,843
‫it's that 0.001 that we used before.

255
00:14:34,670 --> 00:14:36,060
‫We don't want it in meters,

256
00:14:36,060 --> 00:14:39,000
‫because that's not really a readable unit.

257
00:14:39,000 --> 00:14:40,823
‫Instead we want it in kilometers.

258
00:14:42,910 --> 00:14:46,563
‫So now we can go ahead and use out multiplier variable here,

259
00:14:48,060 --> 00:14:51,363
‫give it a save, and let's try it out.

260
00:14:54,450 --> 00:14:57,400
‫So take a look at what we have here in kilometers,

261
00:14:57,400 --> 00:15:00,650
‫so from the previous result, which is 64.

262
00:15:00,650 --> 00:15:05,650
‫That should approximately be 40 miles, so let's send that,

263
00:15:06,560 --> 00:15:08,720
‫and that was pretty close.

264
00:15:08,720 --> 00:15:11,370
‫So 40.2 miles indeed.

265
00:15:11,370 --> 00:15:13,450
‫And so that's our closest tours,

266
00:15:13,450 --> 00:15:18,010
‫and the most farthest away is the City Wonderer,

267
00:15:18,010 --> 00:15:20,630
‫which I think starts in New York or something,

268
00:15:20,630 --> 00:15:24,843
‫and so that is more than 2400 miles away from L.A.

269
00:15:27,630 --> 00:15:30,770
‫If we then set it here to kilometers,

270
00:15:30,770 --> 00:15:33,490
‫then it should be back to getting the value

271
00:15:33,490 --> 00:15:34,583
‫that we had before.

272
00:15:36,960 --> 00:15:38,410
‫Let's put it back to miles

273
00:15:38,410 --> 00:15:41,370
‫because I know that most people watching this course

274
00:15:41,370 --> 00:15:45,093
‫are from the U.S., and so over there they use miles

275
00:15:45,093 --> 00:15:47,320
‫instead of kilometers.

276
00:15:47,320 --> 00:15:50,710
‫So let's save this here as well to our collection

277
00:15:52,300 --> 00:15:57,210
‫get distances to tours from point.

278
00:16:01,940 --> 00:16:04,430
‫So that's it, that wraps up this lecture,

279
00:16:04,430 --> 00:16:08,160
‫and that's all I had to show you about geospatial data.

280
00:16:08,160 --> 00:16:10,730
‫So this video and the last one should have given you

281
00:16:10,730 --> 00:16:13,180
‫a very great overview of how to work

282
00:16:13,180 --> 00:16:16,260
‫with geospatial data in MongoDB.

283
00:16:16,260 --> 00:16:18,900
‫And as I said before there's a ton of possibilities

284
00:16:18,900 --> 00:16:21,647
‫of stuff that you can do in your own applications

285
00:16:21,647 --> 00:16:23,563
‫using this kind of data.

