OK, let's do something a little more advanced with the prompter you will read and next logs and create
a simple dashboard Engine X reverse proxy that was installed at the beginning of the course, he had
the reverse proxy Bafana with Engine X. So every request is going by.
That reverse proxy is being locked into a log file and make it read that using prompts in Loki.
Also in Loki will use what's called the pattern parser, but will go into that.
First, we open up our scraped config file conflict, prompt Obama on our Quanta server and add this
extra section here.
This is a second script config called Engine X. This target is local host.
The job name is Engine X and the path is VAR Log Engine X Star Log.
So going on to Mega-fauna Server, OK, someone might have found a server.
We'll have a look at that folder where the logs are.
So c d the logs and generics ls lh.
There are the log files that Engine X is saving, and you can see that they're accessible via the IDM
group.
So our prompt, our user is already in the IDM group, but if you are using a specific user for prompt,
they'll then make sure that user is in the IBM group so that it can read the logs.
OK, so let's now edit the prompt her config file.
So see the usr local bin Telis L'hygiene that was config prompt.
Tell this so that the nano config from tail y eml.
OK, this is my existing pronto.
Remember, I've explicitly said that two nine nine seven you can let that to zero if you like.
That's the URL that my local prompt tail is sending a sending to a local Loki as existing script config
whose job name is bollocks.
I added the host label there for gharana.
Now I've positioned my cursor where I want to start pasting now going to just copy that part, including
the whitespace control, see right click and a paste case.
So job name and your next set of conflicts targets local host labels and generics.
And that is the path to the log files that will read so VAR Log Engine X style log book save that control
x yes and to its restart restart from tail and check its status.
It looks good active running and I'm not seeing any problems.
OK, so we can now go into Carafano and open up, explore and find a new entry here on the job code
Engine X, so click Engine X and that is the log stream selector job equals Engine X and show logs,
and we can begin to see logs that prompt tail is now pushing into Loki.
OK, so we can see here the file name is VAR Log Engine X Access Log Host Crafar Nut Job Engine X.
There's another one access local Engine X. And if we look at the details of the launch, we can see
that was opposed to the Lokey service.
That's the IP address of my Esquibel server using a low key push method.
If remember, I set my prompt in the mosque will server to go via the Engine X reverse proxy where it
was using the domain name.
And so that IP address there is my actual server that I'm using to make this video and I'm making requests
to the graphical user interface every time I press a button on the Griffon user interface.
Anyway, there's a lot of information in these log lines here that we can query, but this is a good
opportunity to learn a new feature in Loki, and that is the pattern parser.
The pattern parser will allow us to take parts of those log lines and create labels from them.
So, for example, job equals Engine X will pass a pattern over that.
So pipe pattern we're matching a string and putting matches into labels.
So in this pattern here, according to new labels called method and Status, if I look at the log line
here, there's those two hyphens there, there's two hyphens there.
So we could be taking the first property that IP address and putting that into a value or that one.
But we're taking method and status.
So those here, that's the method.
The word post and status is the number two I four.
So copy that line.
Put that into your query to pop pattern.
We're matching that pattern, that whole string and creating two new labels called method and Status.
And if it can match that string and find values to put into method and status, it will create them
as new labels for us.
We'll say that's a shift.
Enter now.
If I look at one of these lines, I've now got two new labels here method and status, so I can now
start using those labels further in my query here.
So, for example, let's count all by time, so can credit graph.
So going to the beginning count over time bracket, what to the end and we'll say for range of one minute
and then we'll close that off with the bracket shift.
And so we now start getting a graph of the zoom to that day.
There we go.
We can start to see the different kinds of methods and status codes that our engineers service proxy
is things.
So status 200 Forget method status two hundred post method.
There was a status four hundred down there instead of two 04 if I zoom in.
To say that section there that looks at the more interesting, these are the kinds of status codes that
we're seeing and that quite common to means okay, but you might get lots of four or four errors and
that means file not found.
You might get sort of a five hundred errors, which means it's a problem with the application running
behind your web server or reverse proxy.
So I'll create a four for error now.
So if I go to the recipe code dot net and it just type in some junk that will return for a four page
not found for a.
Now we'll see that you now end your next job now.
OK, so if I change that query to the last five minutes, I just zoom into that section there.
There is a for for just the red line there.
So that's the four a that I just generated about 10 seconds ago.
Excellent.
So on a busy web server, it's good to see what all the status codes are because if you're suddenly
getting status five hundreds, it will stand out like a sore thumb.
If you see a sudden rise in four iPhones, you're not as a problem as well.
And there are many status codes and you can look those up on the internet what they mean anyway.
So that's good.
So they're looking at the typical log lines that you get from an engineer.
So it's a small sample here familiar on there are many values, so we can see this IP address requesting
that's called remote address is a time that's called time local here.
We don't have a remote user, but you might see that sometimes there is a method that's post.
There is the quest.
That's the path that was being requested from your web server.
There was a protocol payslip one point one.
You'll see different versions of how you stop being requested by move further along.
That's the status code to have seen that Bytes sent zero HTP refer.
We're not saying that day.
You might find that value.
Sometimes I use to be user agent pronto, but when I'm using my browser that I used to be user agent
is usually something more complicated.
Like that Mozilla five app, a web kit, etc. So all those values can be extracted by modifying our
pattern.
OK, so here's an example where I write labels for remote address and time like also copy that string
and will replace the whole load.
And so if I look at the labels, it now says remote address and time local soccer, my queries to refine
on those two values if I needed to.
What I'm going to do is modify this one and add method status back so that you can see that we can use
all those values if we want.
My address time or method was at that position.
Their method and status was at that position.
Their status shift.
Enter.
OK, if I look at the labels, I'm also sing status and method again as well.
So method post.
Now, it's not advisable to create variables for all of these things if you don't actually using them
because it's just not good for performance.
Pretty well, I'm just showing you that it's possible.
Also, you can change the names of the labels anything you like if you prefer my address like that.
For example, it now says from my address like that.
So you got the freedom.
So the Pattern Pass is actually really good and it's actually very fast as well.
In the past, you would use something like rejects in that position, but they say the pattern parser
is now the fastest way of doing this.
So and it looks pretty easy as well.
So we use a pattern passel.
OK, so excellent.
Now I'm just going to get rid of my address there, so I'm not going to use it.
I'm not going to use time local either, but I'm going to create a graph from that, but also group
as well.
So because then I'll use that in a dashboard.
So going back to count over time bracket for one minute and close off that bracket, I'm going to sum
that some until.
So we're creating one line and then I'm going to group by status there.
So I've got a simple graph now that is just showing status codes.
I don't really care about the method, but if I did care, I could just put in method like that and
I've got the label standing using the method and the status.
I'm not going to use it.
Also, another thing that I haven't shown you as well.
You can change the order of this grouping clause by saying some high status and then that's the remainder
of the query.
And so that's the same result.
So that's an option.
If you prefer written like that some by status, then your query that also works for method, some by
status method, some by status method, job, if you wanted to.
But I'll do that.
Now I'm going to use that in a dashboard.
So copy that and let's create a new dashboard that an empty panel select low-key pace that query and
to apply and just save this very quickly, calling it Engine X save and we can just to start down to
15 minutes, for example.
And then we go.
I can add a log panel as well, so I can see the related log files.
So that's at a panel time series or chosen logs.
Low key.
Curly brackets.
Job engineers.
Very good apply.
Position at that time is that for the last five minutes and I want to know something more about that,
for example, I can zoom in or consuming.
Excellent and see the related little clients.
OK, so that's very quickly a basic Engine X dashboard.
I'm going to pause the video, create something a little more complicated.
So anyway, I have go on and buy an extra panel here, which uses the bar gauge there just to create
a summary of the remote addresses and how many times they're making a call to my web server.
That's the query there.
I'm using the bar gauge.
Some can have a time job Engine X pattern remote address for the time range.
I'm using this dollar range instead of one minute I'm using.
The range property there are remote address that means that when I change the time here, the numbers
will be more reflective of how many times in that period the last five minutes we can see that one of
these IP addresses is making a lot of requests to my server, so I could deny that IP address.
If I wanted to put any way, I'll save that save.
Let's go back to the dashboard and I'll just reposition it.
Like so anyway, this dashboard, Jason, here I'll put on my documentation so they can copy and paste
that is down here on the Sample and snakes dashboard, so you could copy that whole lot to the clipboard,
go to dashboards, manage or save that for I go.
Import by panel, Jason Paste that let me go to the copy load, that name already exists, so I'm going
to change it to something else import.
OK, so I've got that loaded so we can see here straight away what's going on with my next reverse proxy
anyway.
Just so that, you know, my Gravano server is under a current dose, so I'm getting a lot of junk actually
being sent to the server.
We can see that down here.
So if your Gryphon, a server, is on the internet, there are possibilities you might start getting
dosed if someone wants to detox you.
So I'm using digital lotion.
Digital Lotion has an ebook firewall.
So, for example, on a networking under firewalls, you can create a firewall called anything you like.
Set your inbound and outbound roles and you can apply it to a droplet.
So, for example, I can apply it to my grandfather droplet, but I've already done that so far.
Just go backwards and modify my existing one.
Right now, I have all IP version four enabled for hACE2 TPS, so I'm just going to edit that rule.
I'm going to delete that rule and just have those two explicit I-Pace that are allowed to query you
to be a smug, often a server that's port four four three.
So I'll save that now.
If I go back into Jovana, we'll start to see that these numbers will start dropping down.
So fast forward this video for a moment.
So all these extra IP addresses on the right here are all being blocked, except for the two drive explicitly
allowed in my firewall.
One of those is my server that I'm creating this video from, and the other one is my MySchool.
So OK.
So you can see now that the graph is going down.
So I go, that's one of the things that serves only to get divorced occasionally.
Excellent.
So if I looked at the last one minute see with this case, we can see that there are less remote addresses
and eventually just be just the two, which are my two IP that I've explicitly allowed.
Excellent.