WEBVTT 00:00.330 --> 00:04.410 Now we'll look at enticing queries and look at the log in the graph panels together. 00:04.470 --> 00:06.180 First will prepare some queries. 00:06.390 --> 00:14.990 So explore and I'll create a dashboard that shows VAR logs for both of my hosts same time module and 00:15.070 --> 00:16.220 farm Sherlocks. 00:16.260 --> 00:19.740 OK, so that's the logs that we'll see in our dashboard. 00:19.800 --> 00:24.270 Very simple query job because VAR Logs is going to save that for after job because VAR logs. 00:24.600 --> 00:28.530 The other one I want is to graph that, so I'll wrap that in. 00:28.530 --> 00:33.480 I count over time one minute and finish that off. 00:34.110 --> 00:35.870 And so now I can grab that. 00:35.910 --> 00:44.220 So I'll use those to query my dashboard if I go now to write a new dashboard, create dashboard, add 00:44.220 --> 00:44.910 an empty panel. 00:44.970 --> 00:47.870 The first one will be a lock down the bottom there. 00:47.940 --> 00:55.920 My low key query was job because bollocks, the Canada and there it is old school VAR logs OK supply 00:55.920 --> 00:59.160 that will create another panel which will use the time series. 00:59.180 --> 00:59.940 That's OK. 01:00.360 --> 01:04.710 Low key and it'll be that query there over time. 01:05.840 --> 01:10.370 One minute, so I'm saying lots of information there, all my log files that have been written on both 01:10.370 --> 01:16.550 servers, both hosts on California and Moscow and overdue sat down to one hour, for example, because 01:16.550 --> 01:17.460 that's pretty good online. 01:17.620 --> 01:18.620 VAR logs again. 01:18.620 --> 01:19.110 Excellent. 01:19.190 --> 01:19.650 OK. 01:19.710 --> 01:26.090 These descriptions here are quite long, so I could actually make those shorter by using a salmon grouped 01:26.090 --> 01:29.960 by options some that I host phone. 01:29.960 --> 01:35.390 I didn't click out of that, and those lines are now slightly shorter because it's not actually showing 01:35.390 --> 01:37.370 tropicals bollocks anymore. 01:37.400 --> 01:38.440 That's just an option you have. 01:38.450 --> 01:41.720 That's just one reason for using some and the grouping down there. 01:41.750 --> 01:45.140 So I'm happy with that apply that I can just reorder this a little bit. 01:46.100 --> 01:52.370 OK, so now if I change my time, filter up there, whatever I see here, they're the related box down 01:52.370 --> 01:53.990 there so I can zoom right into that. 01:54.210 --> 01:58.100 Related logs goes even further and other related logs from both service. 01:58.130 --> 02:03.740 I'll go back to one hour now to add an extra layer of querying code annotation queries over that now 02:03.740 --> 02:05.090 with a nice log launch here. 02:05.150 --> 02:07.640 There are occurrences of invalid use. 02:07.880 --> 02:10.730 I would like to have those highlighted on that graph. 02:11.120 --> 02:14.750 So while it's quite hard to actually see any here, there are likely to be some in there. 02:14.780 --> 02:20.930 So what I can do is create an annotation query that is executed over the dashboard appear so dashboard 02:20.930 --> 02:22.940 settings annotations. 02:23.960 --> 02:25.490 Ad annotation query. 02:26.120 --> 02:32.390 I'm going to call it invalid users, it's going to use a low key data source, it's enabled the color 02:32.420 --> 02:36.770 will be read and my query will be job equals via logs. 02:36.770 --> 02:40.820 Pipe equals invalid use antibodies. 02:40.820 --> 02:41.980 Click out of that, that points. 02:41.990 --> 02:44.900 So I got back to my dashboard and just turn it on and off. 02:45.080 --> 02:49.760 We now start to see some highlights going on here, matching invalid user. 02:49.790 --> 02:54.380 So if I zoom in to them, there's a little arrow just down here. 02:54.440 --> 02:58.040 If you hover over that, it shows you the actual log line that it found. 02:58.670 --> 02:59.660 I can see that again. 03:00.170 --> 03:04.760 And these are all different logging attempts on my my secure server, mostly. 03:05.240 --> 03:07.310 So this is normal for a server on internet. 03:07.310 --> 03:09.200 Automatic scripts will try and log into your service. 03:09.390 --> 03:11.110 OK, so straight away, that's pretty good networks. 03:11.120 --> 03:14.030 Already, I can see if I zoom out to one hour. 03:14.060 --> 03:17.900 There are a lot of attempts to log into my servers going on, so you might be happy with that graph. 03:17.900 --> 03:24.330 But actually something about the syslog log by Zoom to these ones, for example, and I'll find one. 03:24.350 --> 03:29.540 OK, so these log lines here, Bafana Lokey like same day level info, et cetera. 03:29.570 --> 03:36.980 Query equals job VAR logs, invalid user So what's going on here is when you enter log you out queries, 03:37.010 --> 03:39.770 they are all actually saved into the syslog as well. 03:39.770 --> 03:42.310 So you can see here that says they have boxes log. 03:42.320 --> 03:46.170 So any query I create is also being logged, so they're also being matched. 03:46.200 --> 03:52.310 So for look at these again, most of those entries that we see, they're actually just me typing queries 03:52.310 --> 03:54.500 through logical query logs. 03:54.600 --> 03:55.820 Valid user down here. 03:55.850 --> 03:59.180 Query VAR logs invalid user by zoom out. 04:00.430 --> 04:07.480 I actually want to see these kinds of queries where it says Invalides, so I need to refine my annotation 04:07.480 --> 04:08.620 query a little bit further. 04:08.650 --> 04:09.940 Let's zoom in into these ones. 04:10.090 --> 04:12.820 Further, let's find something in there. 04:13.020 --> 04:14.860 OK, I want to look at those ones bit further. 04:14.890 --> 04:16.250 If I look at that one there. 04:16.900 --> 04:23.950 Query job via logs, invalid user that line down the bottom their job VAR logs showing invalid user. 04:24.730 --> 04:28.590 I'll modify the filter to exclude something else returned in that line. 04:28.600 --> 04:35.530 So something that might be useful to exclude could be where it says level. 04:35.530 --> 04:36.550 It calls in for this. 04:36.550 --> 04:40.260 So I'm going to exclude level equals info from the query. 04:40.270 --> 04:45.460 So going back into annotation settings there annotations invalid user. 04:46.150 --> 04:51.820 I would find this filter to be not a class two level because in fire. 04:51.910 --> 04:57.550 So just click out of that so that it binds back to the dashboard now for Zoom back out to one hour, 04:57.600 --> 05:00.210 not seeing as many annotations now as before. 05:00.220 --> 05:05.300 So the annotations that I'm seeing now are going to be more explicit to the type I'm looking for. 05:05.320 --> 05:11.290 They are the actual invalid user log in attempts to link back out to three hours, for example, they 05:11.290 --> 05:12.310 go right back here. 05:12.670 --> 05:18.370 So within those highlighted annotations there, none of those are actually the local queries that are 05:18.370 --> 05:21.460 entered when I'm actually experimenting with the queries. 05:21.580 --> 05:27.580 So be aware that that any query you type into the Explore tab is also being logged into the syslog. 05:27.580 --> 05:29.740 Log on the server with this background. 05:29.740 --> 05:30.820 That's Mega-fauna server. 05:30.850 --> 05:32.140 So I I can save that. 05:32.200 --> 05:40.510 I can call that my VAR log stable site and look at that over six hours if I want, or even over 15 minutes. 05:41.540 --> 05:41.960 Excellent.