Dark Arts
  • index
  • BUY ME A BOOK
  • 🪄Dark Magic
    • Pentesting
      • Industry Methodologies
    • Scopes of Testing
    • Reconnaissance
      • Passive
        • WHOIS
        • DNS
          • nslookup
          • dig
        • WAF
        • Subdomain
        • Google Dork
        • Misc. Techniques
        • Leaked Passwords
      • Active
        • Browser & Plugins
        • ping & traceroute
        • fping
        • telnet & netcat
        • DNS
          • Zone Transfer
          • DNS Amplification DDoS Attack Breakdown
        • Misc. Techniques
    • Vulnerability Assessment
    • Attack Types
  • 🕷️Aragoogs Nest
    • Web Application Overview & Security
      • Security Testing
      • Common Threats & Risks
    • Web Application Architecture
      • Technologies
    • HTTP/S
      • Message
      • Request
      • Response
        • Status Code
    • Crawling/Spidering
  • 🧪Potions
    • Web Browsers
    • Computer Networking
      • Network Protocol
      • Packets
      • OSI Layer
        • Layer 3: Network
        • Layer 4: Transport
      • DNS
        • Primary-Secondary
        • Local Name Resolution
        • Domain Hierarchy
        • FQDN
        • Lookups
        • DNS Resolution
        • DNS Records
        • Security: Attack-Defense (Default)
  • 🎆Spells
    • 📜Linux Scroll
    • 📜WebShell Scroll
    • git
      • Attacks + Vulnerabilities
  • 🖼️Flaws w/ Magical Frameworks
    • Windows
      • In a Nutshell
      • CVE-2019-0708: BlueKeep
      • CVE-2017-0144: EternalBlue: MS17-010
      • Attacking Services
        • MS IIS - WebDAV
        • SMB
        • HTTP File Server (HFS)
        • Apache Tomcat Web Server
        • RDP
        • WinRM
      • File System Vulnerabilities
      • Credential Dumping
        • Password Search in Windows Configuration Files
        • Mimikatz
        • Pass-the-Hash Attack
    • Linux
      • In a Nutshell
      • CVE-2014-6271: Shellshock
      • Attacking Services
        • FTP
        • SSH
        • SAMBA
        • SMTP
        • RSYNC
      • Dumping Hashes
  • 🌼Marauder's Boost
    • Privilege Escalation
    • Windows PrivEsc
      • Windows Kernel Exploit
      • Bypassing UAC
      • Access Token Impersonation
    • Linux PrivEsc
      • Linux Kernel Exploit
      • Misconfigured Cron Jobs
      • Exploiting SUID Binaries
      • shells
      • File Permissions
  • ☠️Death Eaters
    • Post Exploitation
      • Windows
      • Linux
  • 🪄OLLIVANDERS
    • nmap
      • Host Discovery
      • Port Scan
      • Service & OS
      • NSE
      • Firewall/IDS Evasion
      • Scan Optimization
      • Misc. Methods
    • ffuf
    • Hydra
    • Metasploit Framework
      • Architecture
      • Must to Know
      • msfvenom
      • Auxiliary Modules
      • Service Enumeration
      • Vulnerability Scanning
      • Imports
      • Automating
    • Vulnerability Scanners
    • Wireshark
  • 🚂Platform 9(3/4)
    • Auth-Auth
      • Authentication
        • Password-based Authentication
        • Basic Authentication
        • Multi-factor Authentication
        • Access Token
        • Token-based Authentication
          • JWT
          • OAuth 2.0
    • Secure Headers
      • Content-Security-Policy (CSP)
    • Cryptography
      • Caesar Cipher
  • ⛲Port Pensieve
    • Enumeration
      • SMB & NetBIOS
      • SNMP
    • Wordlists
  • 🔆DUELS
    • Pivoting
    • SMB Relay Attack
  • 🗺️Marauder's Map
    • Web Application Pentesting
    • API Pentesting
      • GraphQL
        • Primer
    • Mobile Application Pentesting
  • 🎧SIDE CHANNEL
    • Side Channel Analysis
    • Timing Side-Channel Attacks
      • Vulnerable Login
  • 🥃Sky
    • Cloud Basics
    • Cloud Management
      • Shared Responsibility Model
    • Using Cloud Resources
      • Monitoring & Alerts
      • Identity & Access Management
      • Scalability & Availability
      • Solution Design
    • Cloud Providers
    • Cloud Security & Regulatory Compliance
      • Resource Protection
      • ICCA: Cloud Security & Regulatory Compliance
    • ICCA Preparation
      • Knowledge Tests
      • Lab
  • 🔷Obsidian
    • Pentest Engagement
      • Scoping
    • Pentest Ethics
      • Rules of Engagement
    • Auditing Fundamentals
      • Process/Lifecycle
      • Pentest & Security Auditing
      • GRC
      • Standards, Frameworks & Guidelines
      • From Audit to Pentest
  • 📡THREAT INTEL
    • Threat Intelligence
    • Tool Dump
  • 📱Anything-Mobile-IoT
    • Firmware
    • Firmware Analysis
      • Example: CVE-2016-1555
    • Firmware Installation/Flashing
  • 🎉Mischeif
    • Social Engineering
    • Phishing
      • GoPhish
    • Pretexting
Powered by GitBook
On this page
  • Crawling
  • Spidering
  1. Aragoogs Nest

Crawling/Spidering

Crawling

Crawling is a passive process of mapping a web application by following links, submitting forms, and navigating accessible areas. Tools like Burp Suite’s passive crawler help understand its structure.

Spidering

Spidering is an active, automated process to discover new URLs, starting from seed links. It is a noisier method of information gathering.

PreviousStatus CodeNextWeb Browsers

Last updated 20 days ago

🕷️