Active Information Gathering

Web Servers

• Web servers are directly involved in any web application's operation. Some of the most popular are Apache, Nginx, and Microsoft IIS, among others.

• If we discover the webserver behind the target application, it can give us a good idea of what operating system is running on the back-end server. For example, if we find out the IIS version running, we can infer the Windows OS version in use by mapping the IIS version back to the Windows version that it comes installed on by default.

• Although this is usually correct when dealing with Windows, we can not be sure in the case of Linux or BSD-based distributions as they can run different web server versions in the case of Nginx or Apache.

• We need to discover as much information as possible from the webserver to understand its functionality, which can affect future testing. For example, URL rewriting functionality, load balancing, script engines used on the server, or an Intrusion detection system (IDS) in place may impede some of our testing activities.

HTTP Headers

• By examining the HTTP headers we can learn about the web server.

• We can use cURL to get the header (curl -I <Target>)

  • Server: This header can provide us with the webserver and its version.

  • X-Powered-By header: This header can tell us what the web app is using. We can see values like PHP, ASP.NET, JSP, etc.

  • Cookies: Cookies are another attractive value to look at as each technology by default has its cookies.

• Other available tools analyze common web server characteristics by probing them and comparing their responses with a database of signatures.

• Whatweb recognizes web technologies, including content management systems (CMS), blogging platforms, statistic/analytics packages, JavaScript libraries, web servers, and embedded devices.

  • We can scan a website by providing the aggression level and the -v flag to make it verbose: whatweb -a3 https://www.facebook.com -v

• Wappalyzer is a browser extension. It has similar functionality to Whatweb, but the results are displayed while navigating the target URL. They also have a website where you can provide them with a URL and get the data.

• WafW00f is a web application firewall (WAF) fingerprinting tool that sends requests and analyses responses to determine if a security solution is in place.

  • wafw00f -v https://www.tesla.com

• Aquatone is a tool for automatic and visual inspection of websites across many hosts and is convenient for quickly gaining an overview of HTTP-based attack surfaces by scanning a list of configurable ports, visiting the website with a headless Chrome browser, and taking a screenshot.

  • To Install

    • Download the latest release, https://github.com/michenriksen/aquatone/releases

    • Then move it to the the bin directory (mv aquatone /usr/bin/)

Zone Transfers

• The zone transfer is how a secondary DNS server receives information from the primary DNS server and updates it.

• The master-slave approach is used to organize DNS servers within a domain, with the slaves receiving updated DNS information from the master DNS.

• The master DNS server should be configured to enable zone transfers from secondary (slave) DNS servers, although this might be misconfigured.

• If we manage to perform a successful zone transfer for a domain, there is no need to continue enumerating this particular domain as this will extract all the available information.

Manual Approach

• Identify the name servers: nslookup -type=NS <Domain Name> <IP Address>

  • Example: nslookup -type=NS zonetransfer.me

• Perform the Zone transfer using -type=any and -query=AXFR parameters: nslookup -type=any -query=AXFR <Domain Name> <Name Server>

  • Example: nslookup -type=any -query=AXFR zonetransfer.me nsztm1.digi.ninja

Gobuster

• Gobuster is a tool that we can use to perform subdomain enumeration.

• Sometimes we can discover some naming conventions that an organization uses. For example, lert-api-shv-<Number>-sin6.facebook.com

• We can use Gobuster in such cases to find more subdomains the follows the same naming conventions.

• First we create a wordlist with the pattern we found. For example,

  • lert-api-shv-{GOBUSTER}-sin6

  • atlas-pp-shv-{GOBUSTER}-sin6

• After that we run Gobuster with the following options:

  • dns: Launch the DNS module

  • -q: Don't print the banner and other noise.

  • -r: Use custom DNS server

  • -d: A target domain name

  • -p: Path to the patterns file

  • -w: Path to the wordlist

  • -o: Output file

  • Full Command: gobuster dns -q -r "<Name Server>" -d "<Domain Name>" -w "<Wordlist>" -p <Pattern File> -o "<Output File Name"

    • Example: gobuster dns -q -r "d.ns.facebook.com" -d "facebook.com" -w "numbers.txt" -p ./patterns.txt -o "gobuster_facebook.com.txt"

Virtual Hosts

Introduction

• A virtual host (vHost) is a feature that allows several websites to be hosted on a single server. There are 2 ways IP-based virtual hosting or Name-based virtual hosting.

• During our subdomain discovering activities, we have seen some subdomains having the same IP address that can either be virtual hosts or, in some cases, different servers sitting behind a proxy.

IP-based virtual hosting

• For this type, a host can have multiple network interfaces.

• Multiple IP addresses, or interface aliases, can be configured on each network interface of a host.

• The servers or virtual servers running on the host can bind to one or more IP addresses.

• This means that different servers can be addressed under different IP addresses on this host.

• From the client's point of view, the servers are independent of each other.

Name-base virtual hosting

• The distinction for which domain the service was requested is made at the application level. For example, several domain names, such as admin.inlanefreight.htb and backup.inlanefreight.htb, can refer to the same IP.

• Internally on the server, these are separated and distinguished using different folders. For example, the vHost admin.inlanefreight.htb could point to the folder /var/www/admin. For backup.inlanefreight.htb the folder name would then be adapted and could look something like /var/www/backup.

Enumerating vHosts

• Wordlist: SecLists/Discovery/DNS/namelist.txt

• This manual script can be used to do the fuzzing manually by using curl

  • cat <Word List> | while read vhost;do echo "\n********\nFUZZING: ${vhost}\n********";curl -s -I http://<IP Address> -H "HOST: ${vhost}.<Domain Name>" | grep "Content-Length: ";done

  • Example: cat ./vhosts | while read vhost;do echo "\n********\nFUZZING: ${vhost}\n********";curl -s -I http://192.168.10.10 -H "HOST: ${vhost}.randomtarget.com" | grep "Content-Length: ";done

• We can also automate it using ffuf:

  • ffuf -w <Word List> -u <IP Address to Fuzz> -H "HOST: FUZZ.<Domain Name>" -fs <Size to Filter>

  • Example: ffuf -w ./vhosts -u http://192.168.10.10 -H "HOST: FUZZ.randomtarget.com" -fs 612

Crawling

• Crawling a website is the systematic or automatic process of exploring a website to list all of the resources encountered along the way.

• Many different tools can be used to crawl a target an example of these are using ZAP, BurpSuite, or ffuf.

Sensitive Information

• It is typical for the webserver and the web application to handle the files it needs to function. However, it is common to find backup or unreferenced files that can have important information or credentials.

• A good wordlist for finding such files would be the raft wordlist, SecLists/Discovery/Web-Content/raft*

• We can combine folders and sensitive files using ffuf: ffuf -w ./folders.txt:FOLDERS,./wordlist.txt:WORDLIST,./extensions.txt:EXTENSIONS -u <Domain>/FOLDERS/WORDLISTEXTENSIONS

  • Example: ffuf -w ./folders.txt:FOLDERS,./wordlist.txt:WORDLIST,./extensions.txt:EXTENSIONS -u http://192.168.10.10/FOLDERS/WORDLISTEXTENSIONS