WEBVTT 00:00.770 --> 00:06.410 Another tool for crawling websites and emails and whatnot is Spider Foot. 00:06.410 --> 00:11.060 Now Spider Foot has a web site that you could use. 00:11.060 --> 00:14.990 Also, it used to have a free version, however they took it away. 00:14.990 --> 00:20.840 However, the GitHub page is still available and still free. 00:21.620 --> 00:24.260 So depending on how much. 00:25.040 --> 00:29.570 You need in terms of scanning, you may want to go with the paid version. 00:29.570 --> 00:34.460 However, the free version I found to be to be really useful. 00:35.010 --> 00:36.930 So essentially spider Foot does it. 00:37.140 --> 00:40.800 It crawls here and it for different things. 00:40.800 --> 00:42.630 You could do email addresses. 00:42.930 --> 00:48.180 You could do websites which is what we're going to demonstrate in this video. 00:49.090 --> 00:50.680 And it does really great. 00:50.680 --> 00:54.280 Reconnaissance by spidering the internet for that information. 00:55.650 --> 01:05.130 So we could find over github.com forward slash s m I c a l l f for slash spider foot. 01:05.130 --> 01:09.090 Or you can type in spider foot git and it should take you to this page. 01:09.800 --> 01:11.120 Now it's pretty simple. 01:11.120 --> 01:12.800 This is a Python program. 01:12.800 --> 01:19.430 If we scroll down in here, we can see the web interface that that we that it has. 01:20.160 --> 01:24.480 And if we go down in here, we can see the stable build package release. 01:24.480 --> 01:30.330 The easiest thing to do is just click this little box here that'll copy all of this here. 01:30.900 --> 01:35.700 And then what you want to do is you want to open a terminal, paste it and run it. 01:35.700 --> 01:42.000 And what that's going to do is it's going to w git is going to download the spider program. 01:42.480 --> 01:46.080 Then tar the tar file is going to. 01:47.330 --> 01:51.380 Uh, converted CD goes to the spire foot directory. 01:51.380 --> 01:56.060 Pip3 install is going to install the requirements for spire foot, and then it's finally going to run 01:56.060 --> 02:04.070 it in the command to run it one when you want to run it again later is going to the spider foot folder 02:04.070 --> 02:05.390 CD Spider foot. 02:05.390 --> 02:08.090 You can just tab to auto complete once you're in there. 02:08.090 --> 02:17.270 Python three space period four slash sf dot p y dash l and then the address there. 02:17.630 --> 02:21.230 And what this is going to do is it's going to open up a. 02:23.320 --> 02:26.410 Your local page here. 02:26.410 --> 02:33.880 So on the browser it's going to open this 127.0.0.1 port 5001. 02:34.760 --> 02:37.010 And this is going to be the spider foot page. 02:37.010 --> 02:41.330 Now I'm running a scan here to give a demonstration because it can take a while. 02:41.330 --> 02:45.260 However, if I go to new scan here, we can type in a scan name. 02:45.260 --> 02:47.810 I could do test two for example. 02:48.140 --> 02:52.520 And then what you want to scan, do you want to scan a domain IP address? 02:52.520 --> 02:56.210 IPv4 IPv6 hostname. 02:56.210 --> 02:57.710 Subdomain subnet. 02:57.710 --> 02:59.030 Bitcoin address. 02:59.030 --> 02:59.750 Email address. 02:59.750 --> 03:00.530 Phone number. 03:00.530 --> 03:02.570 Human name, username. 03:02.570 --> 03:04.250 Network ASN. 03:04.250 --> 03:08.900 So we see there's a lot of different ways that we can scan for information. 03:09.710 --> 03:15.710 Once you put your target scan in any of these criterias then we could go down in here. 03:15.710 --> 03:17.240 We could see these different scans. 03:17.240 --> 03:18.980 All you could scan everything. 03:18.980 --> 03:25.730 However, if this is if you're doing a pen test, I wouldn't recommend doing all because it it can be 03:25.730 --> 03:26.660 a little noisy. 03:26.660 --> 03:29.690 And this is also a very slow scan. 03:29.690 --> 03:35.150 You could do footprinting, you could do investigating, you could do passive scanning. 03:35.150 --> 03:39.890 If you're doing a pen test, you probably want to do a passive one first. 03:39.890 --> 03:44.000 Kind of get an idea without sending off too many alerts. 03:44.890 --> 03:47.770 So once you choose your criteria, you choose your target. 03:47.770 --> 03:52.660 You click run Scan now and they will start the scan. 03:52.660 --> 03:54.910 Now I'm going to go to my scans folder here. 03:54.910 --> 03:56.410 And we're going to take a look at this one. 03:56.410 --> 03:59.860 We can see this one is still running I'm scanning Yahoo. 04:00.340 --> 04:02.710 Let me give you an idea of what it looks like though. 04:02.710 --> 04:06.940 So if I click on test here we can see it pulling up. 04:07.030 --> 04:12.100 Um a nice little graphical chart here of different information that it's finding. 04:13.030 --> 04:15.100 And we could do correlation. 04:15.250 --> 04:19.900 We could do we could browse so we could see affiliate email addresses. 04:19.900 --> 04:27.400 We see it found 65 unique elements App store entries, domain who is etc. etc.. 04:27.400 --> 04:30.490 So uh, very cool program. 04:30.490 --> 04:32.590 We can see all the graph settings. 04:32.590 --> 04:35.800 We could take a look at the scan settings and log files. 04:36.790 --> 04:37.870 So again. 04:38.690 --> 04:41.330 This microphone makes things really easy. 04:41.750 --> 04:45.440 Uh, what I like to do is run a spider foot scan for certain things. 04:45.560 --> 04:49.820 Whether it's a name, an email address, a domain, whatnot. 04:49.820 --> 04:53.930 And while that's running, since it is probably going to take a while, this has been running for a 04:53.930 --> 04:55.100 couple of minutes already. 04:55.760 --> 04:58.220 I like to go out and do my other checks. 04:58.220 --> 05:05.180 Maybe it's a manual check using Metamucil or I'm sorry, not Metamucil, but, uh, Google dorks. 05:05.180 --> 05:12.140 I might use some like Sherlock or other programs while this is running, uh, to see if I could find 05:12.140 --> 05:13.160 additional information. 05:13.160 --> 05:19.490 And once those checks and scans finish, I can compare notes on these different scans and see if there 05:19.490 --> 05:23.330 see if it found the same information and verify my results. 05:23.330 --> 05:25.370 So again this is spider Foot. 05:25.370 --> 05:34.790 And you can find it over on the GitHub page github.com forward slash s m I c a l e f forward slash spider 05:34.790 --> 05:35.360 foot. 05:35.360 --> 05:43.580 And again this is a really cool tool for pretty much anything usernames, email addresses, domains, 05:43.580 --> 05:47.030 bitcoin addresses, etc. so thank you for watching. 05:47.030 --> 05:48.200 I'll see you in the next video.