Misc. Techniques

Host (cmd)

The host command in Linux is used to perform DNS (Domain Name System) lookups. It helps retrieve information about domain names, IP addresses, and DNS records.

host example.com #Domain Lookup
host 1.1.1.1 #Reverse Lookup
host -t MX example.com #Record Lookup

Hunting Hyperlinks

In passive reconnaissance, examining a website's hyperlinks is a valuable method to gather information without interacting with the target system. By analyzing the links on a webpage, you can discover: Hidden of Lesser Known Directories, Internal and External Resources, Subdomains and other related websties, Potential Login pages or Admin panels.

robots.txt

robots.txt: The robots.txt file is used by website owners to control how search engine crawlers access and index their site. It specifies which parts of the site should not be crawled or indexed using the "Disallow" rule. This is useful for keeping certain files or directories private or hidden from search engines, though it doesn't prevent access directly.

User-agent: *
Disallow: /admin/
Disallow: /private-data/

# In this example, all search engine bots (User-agent: *) are instructed not to crawl the /admin/ and /private-data/ directories.

sitemap.xml

The sitemap.xml file provides search engines with a structured list of all the URLs on a website, helping them efficiently crawl and index the site's content. It ensures that important pages, especially those that might not be easily accessible through internal links, are discovered by search engines.

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/</loc>
    <lastmod>2023-01-01</lastmod>
    <changefreq>monthly</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://www.example.com/about/</loc>
    <lastmod>2023-01-01</lastmod>
    <changefreq>yearly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

## This example shows URLs of the website with details like modification date, frequency of changes, and importance for indexing.

Technology Profiler

A technology profiler is a tool or technique used to identify the technologies, frameworks, and software stacks a website or application is built upon. It helps penetration testers by providing crucial insights into the target's technology landscape, such as the web server, programming languages, CMS, databases, and other components.

Browser Plugins

whatweb (cmd)

WhatWeb is a tool that identifies web technologies used by websites, such as CMS platforms, web servers, and JavaScript libraries. It uses over 1800 plugins to recognize various components, including version numbers, email addresses, and SQL errors. WhatWeb offers different aggression levels, allowing it to be either fast and stealthy or thorough but slower, depending on the need. It can identify technologies even when common identifiers are removed, making it useful in penetration testing and website profiling.

whatweb --help
whatweb --version
whatweb example.com

Copy Webstie

HTTrack is a website copier tool that allows users to download an entire website, including its structure, for offline browsing. It mirrors the website by downloading HTML pages, images, directories, and other files, preserving the original site’s structure.

  • Penetration testers can analyze a website's content offline without alerting the target.

  • HTTrack allows pentesters to explore hidden directories, file structures, and configuration files that might reveal sensitive information.

  • By mirroring the site, pentesters can better understand the website's architecture and identify potential entry points for attacks.

Netcraft

Netcraft is a comprehensive tool for website footprinting, providing key insights into websites' infrastructure, ownership, and security. It combines WHOIS data, SSL/TLS certificate details, and a technology profile of the target site, making it a valuable resource for security professionals and analysts. With Netcraft, you can gather information such as server details, technologies in use (like CMS or web server), and domain ownership, all in one platform. This makes it ideal for basic website investigations and in-depth profiling.

theHarvester

theHarvester is a tool used in the reconnaissance stage of security assessments to gather open-source intelligence (OSINT). It collects email addresses and hostnames from various public sources, aiding in identifying a domain's external threat landscape.

Installation: https://github.com/laramies/theHarvester/wiki/Installation

theHarvester -d example.com -b google
PreviousGoogle DorkNextLeaked Passwords

Last updated