Assisted directory brute forcing

Very frequently during a web application assessment a pentester may begin by fingerprinting what web frameworks and libraries are used by a given application.  Possibly by running a tool such as blindelephant or whatweb.  Commonly though the tester may just notice a unique identifier in the web traffic or URL.  Consider the following: 

So simply looking thru the initial page you can generally see unique items that will reveal various frameworks or CMSs.  Alternatively, you could find various unique pieces of the web application because of your initial OSINT or recon.  Like this (except your should append site:<target domain> to the search):

So what does that have to do with anything?  Well, one of my favorite web vulnerabilities is weak permissions on sensitive files.  Think of things like phpMyAdmin, TinyMCE, or FCKeditor etc.  Web servers are very rarely setup correctly, and misconfiguration will most likely lead to compromise or at least information leakage.  I found myself taking the unique indicators within web applications and searching for the projects on Github.  Once I located the github project I was taking the git repo to create a directory and file list for brute forcing later.  I believe in automating anything I have to do manually more than five times.  So that is what I did.  You can find the python script here.     

Run the script like this:

Once done use the output file with your favorite directory brute forcing tool.  I like dirs3arch, so for that tool it would look like this:

I have found this technique very useful, hopefully you do as well.  In the future I intend to code the functionality into a full featured directory brute forcing tool such as dirs3arch, goBuster, or dirb.