SecurityTrails Blog

Even if some people believe that robots and automation will replace the entire workforce and machines will do everything for us, the truth is that automation helps us to be more productive and work more efficiently. It relieves us from the most tedious and monotonous tasks in our daily work and lives.

Automation is already all around us. There's little to nothing we haven't found a way to automate. We have robot vacuums cleaning our homes, there is no way to remember the last time we had to leave our house to pay bills, many of us now use voice-to-text typing, and self-driving cars are going mainstream.

Automation has found its way into every environment, from industries to daily administrative tasks. If we found a way to automate the mere typing of words, we naturally found a way to **automate low-level, repetitive, and time-consuming security tasks**.

Hackers waste no time trying to access devices and networks to steal information, exploit application vulnerabilities, even launch ransomware attacks. Likewise, we shouldn't be wasting our own time with tedious, repetitive security tasks and manually analyzing and responding to each and every alert and incident.

Many security tasks can be automated, and in turn made more time-effective, leaving time for security teams to work on deeper analysis and higher-level tasks. Security monitoring, detection and incident response, among other tasks, can be automated and frequently are.

Before we go deeper into security automation tools, let's start with what security automation is, what it means in the current threat landscape, how to get the most out of automation, and more.

What is security automation?

Security automation is the **automatic execution of security tasks without human intervention**. This includes any security action involved with detecting, analyzing, preventing or remediating cyber threats that is automated, therefore, machine-based and contributes to the overall organization's security posture and plays an active, or better yet, proactive role in future security strategies.

Before automation, many tedious security tasks were performed by practitioners and analysts who went through numerous alerts, analyzed, and decided whether and how to respond to them.

With security automation, security teams are now equipped with a solution that can work for them and take on all the security tasks that took time from security professionals. Valuable time that could be used for involvement in more strategic activities and work on proactive security measures.

There are a few signs that tell us a security task should be automated:

**Repetitive, everyday tasks**: Routine tasks that are done on a daily or otherwise regular basis, such as going through security alerts and analyzing them to differentiate between false positives and genuine alerts and potential threats.

**Tiresome, monotonous tasks**: Security tasks that always follow a similar set of rules and steps. For example, a security incident involving a flagged email and a potential phishing attempt would require analysts to manually check URLs, check domain owner information, IP geolocation, etc.

**Time-consuming tasks**: Security tasks such as correlating data and finding patterns in collected data can take up a lot of time and that time can be invaluable in uncovering suspicious activity before any real attacks happen.

What are the benefits of security automation?

In general, security automation has been predominantly utilized by SOC teams and security teams for quite some time. Its introduction addresses a couple of key challenges these teams face—best reflected through the benefits it has brought them.

Combats alert fatigue

Over the past several years, we've added a lot of detection tools and monitoring technologies to our workflows. Keeping in mind the amount of tools, the attack surface they monitor and the number of security incidents these tools respond to, many of which are just false ...

If you or any friend, family member, or acquaintance has ever used an email account, chances are that at some point, they've received a phishing email. And while during the early stages of the internet, such deception usually looked so fake and misaligned that you could spot it right away as an attempt to mislead you, this isn't so true anymore. === An accustomed internet user with a keen eye might spot the fakery in the above picture, mainly because of the suspicious-looking email address. But its look and feel are very similar to that of an actual, legitimate message.

This design task (worthy of a craftsman web designer) used to be a time-consuming endeavor that could take several hours to get right, but that's no longer the case with the arrival of complete frameworks that make today's "Deception as a Service" ecosystem feasible. Today we're taking our previous article Phishing Toolkit: Top 20 Best Phishing Tools one step further, by highlighting one of the tools listed: Gophish, the open-source phishing framework. ## Red Team Phishing Exercise For this post, what we propose is for you to take the lead in a red team exercise for your own company.

We'll target fake characters, giving you an example of how you can do it to yourself—by using our own apps and infrastructure to simulate the test, just as a cyber espionage team would do. By doing this we intend to show you today's tool-goodies, but more importantly, we also want to show you a great way to test your own company's behavior against this kind of attack.

Lastly, we'll explore how easy it is to clone any web page and corporate email template to make it look **almost real** , with an emphasis on the **real** part, as these techniques and frameworks have evolved to the point where you could easily pretend to be anyone or anything, and not just the usual social networks and global free email services that are usually targeted. ## Phishing 101 Phishing is one of the most common types of cybercrime that may require getting several pieces together to work, with their different flavors depending on the focus of the attack (such as so-called "whaling" when specifically targeting very high profile executives or government individuals).

In comparison with other types of attacks, such as its little, but more widespread brother the scam, like the infamous Nigerian scam, you may find phishing attacks more on the technical side. This attack requires not only a good-looking and carefully crafted email to work, but also a functioning link that leads the deceived users to its real core, the prize-collecting webpage. While this may be at the frontier of the phishing definition, it is somewhat unusual for people to "just find" phishing websites.

Instead, they're usually cleverly conducted toward them by malicious actors who commonly host fake websites in awkward domain names and URL paths. By creating clever ways to deceive people into entering the link within an email, victims fall into the trap and end in a hostname similar to the one in the image above. This link will contain the necessary files to appear as the real service the victims believe they are in, working in conjunction with methods to store and forward gathered information to the attacker. ### Getting the software Gophish is available here for the three main operating systems, Windows, Linux, and MacOS.

Once downloaded, and after comparing the hash checksum to the obtained zip file we can execute the binary to start the application. ## Running Gophish When running, Gophish, by default will start two listeners on TCP ports 3333 for the admin interface and 80 for phish-webpage requests. Once the process has started, we can invoke the browser on and enter the admin interface using the admin:gophish credentials.

If desired, you can create and configure your own set of certificates or place your own in case you have any for this purpose. In any event, the installation documentation may help. Ok, we're inside! Le...

Top 10 Internet Search Engines Used by Hackers . We all know what internet search engines are, and why they're used. Even when you want to search for the most mundane thing ever, you can go to Google, Bing, Safari, or any of the other traditional web...

We all know what internet search engines are, and why they're used. Even when you want to search for the most mundane thing ever, you can go to Google, Bing, Safari, or any of the other traditional web browsers. But what if you need access to information not usually found on those popular search engines?

Security professionals often need to quickly look up and correlate data during Osint, reconnaissance, discovering vulnerabilities, finding security breaches in networks, and more. Fortunately, there are search engines in their arsenal of tools that are designed to be used by hackers and professionals. These can provide valuable data for their security operations.

Search engines used by security professionals, are valuable tools for both red teams and blue teams, as well as security researchers, analysts, and others. These tools can help in finding exposed devices, tracking threats, preparing for spear phishing simulations, and much, much more in the area of security operations.

The 9 best search engines for Infosec Professionals

We have tested and aggregated a list of the best search engines often used by security professionals—and anyone wanting to perform a deeper search than those offered with traditional search engines like Google. And we’re not talking about private search engines; we've already compiled a list dedicated to the best private search engines to ensure data privacy while online.

This list identifies search engines that help security professionals obtain information about connected devices, private information on individuals (in a more curated manner than that provided by traditional search engines), breached credentials, and more. Let's start here:

1. Shodan

Being called both "the search engine for hackers" as well as "the world's first search engine for Internet-connected devices", it's easy to guess why Shodan is the first logical choice. Shodan is a network security monitor and search engine that indexes data from any type of electronic device that's ever been connected to the Internet. And by any type, we mean it: webcams, routers, servers, smart TVs, refrigerators, traffic lights, heating systems, and much, much more.

Imagine Google, but instead of seeing the usual images next to different websites and their content, on Shodan you see results in various forms including IP addresses, types of devices, country, and of course SSH, Telnet, and HTTP server banners. But the real value of Shodan is in its ability to help both red teams and blue teams.

For the defenders, Shodan can play an important role in providing visibility to their devices and open ports, and even search for device vulnerabilities to some known exploits. Basically, blue teams are able to retrieve Osint data and identify devices they need to secure. Simultaneously, Shodan helps red teams and penetration testers with reconnaissance and information gathering, even going as far as discovering security flaws in target devices.

Shodan also offers one of the best cybersecurity APIs out there for anyone wanting to integrate their services into apps, with great documentation and libraries that make it easy to set up. Some of the Shodan API's features are support for Python, Ruby, PHP, C#, Go, Hackell, Java, Node.js, Perl, PowerShell and Rust, REST API, network alerts, on-demand scanning, and a streaming API used to watch real-time data feeds.

2. Censys

Right next to Shodan, we have Censys. Censys is similar to Shodan in that it monitors every device exposed on the Internet, and in the form of a search engine, provides data and aggregates reports on how devices, websites and certificates are configured and deployed.

Censys constantly collects information on connected devices and internet servers to help you get the most accurate data about any device connected to the internet, along with details on open ports, protocols and valid certificates. It also identifies server versions, unpatched vulnerabilities, routers, OS vers...

How much information about a target can you possibly get? Is there an invisible barrier that delimits when enough information is gathered? What about cross-checking results and looking for differences? Data retrieval results may vary, depending on the location source you're running for the analysis.

Let's answer these questions today, by running DMitry along with our own SurfaceBrowserTM️ enterprise tool. We'll find out what happens when information is extracted from different sources so they can complement each other.

What is DMitry?

While this small tool called DMitry is considered old code, it does have a few useful information gathering tricks up its sleeve. Check out these options:

As we see above, it can do the following tasks:

Perform an IP Who-Is check - placing a determined IP address and getting its owner information

Perform a Who-Is lookup on the domain name (this is self-explanatory)

Extract information about a target on the netcraft.com website

Utilize a subdomain finder to look upon different records available

Search the web for 'low-hanging fruit' information regarding email addresses, like the ones within Who-Is records or start of authority DNS records

Conduct port scanning and banner grabbing regarding discovered open ports

Testing DMitry

We're using the Kali Linux distribution to run this tool but you can download the source code here. Be aware that with the current version (1.3a), there's an issue regarding netcraft.com queries, which doesn't work due to the change of the URL that it points to for conducting hostname queries.

Aside from that, to emulate this inquiry you can access this link, and using this tool can look like the following example:

This check alone brings some additional information about our target in just one place. As we mentioned earlier this feature inside the tool is broken at the time of this writing.

Who-Is against IP addresses and hostnames

"Who-is" (as pronounced) is a protocol running on port TCP/43 that's used to query databases of ownership information regarding internet assets—such as domain names, autonomous systems, and internet protocol addresses block numbers.

Contrary to what one might think, IP addresses also have information regarding reverse pointer records about ownership and several other details (such as name servers), which could be obtained from Who-Is records.

While the truthfulness of this information depends on the owner of the IP range, we often find obsolete information that can be misleading. In some cases, however, it could also be treated as historical information that can be used for competitive advantage in case of a company's attack surface reduction endeavors (forgotten and vulnerable services come to mind).

DMitry, when used with the flag **-i**, will query IP addresses to gather Who-Is information, and if you include a hostname it will solve the corresponding A record. After this, it will query the server and display information as shown:

When the **-w** flag is used, the execution is similar, but information regarding the specified domain name will be shown:

This will search the Who-Is record for the desired domain name and display its results to the screen.

Harvesting subdomains

An interesting feature included with this exploratory software is subdomain retrieval capability; it scraps the entire World Wide Web to search for possible hostnames within a certain specified registered name. This is possible by using the **-s** flag before the domain name:

For the sake of obtaining the most accurate information, we are going to double-check the subdomain surface and see what else we can find regarding this subdomain. This time, we'll do it by using our very own SurfaceBrowserTM️.

Great! The resulting difference is outstanding... but why? Just to understand how DMitry works, we decided to look under the hood to check its source code. Specifically, we'll be looking at the subsearch.c file located inside t...

SecurityTrails got a big gut check at the beginning of last week. One of our Elasticsearch servers was unintentionally left open when an engineer was trying to fix an outage. This caused a series of self inflicted drama.

We notified clients and wrote a comprehensive technical blog post almost immediately. The response from clients and the security community as a whole was positive and encouraging, with the occasional well-deserved eye roll on Twitter.

This brings us to where we are today. To help better understand where we stand, what we've learned and what we plan to do to be better in the future, SecurityTrails co-founders Chris Ueland and Courtney Couch share their thoughts.

From the founders

"This isn't a small thing to us.

I feel a huge sense of responsibility to solve this issue for every company including our own. I am crazy passionate about solving this problem and have spent every day for the past 3 years of my life thinking about it.

We had this issue even though we know the risks, heck - we have 230+ blog posts surrounding the topic educating others on our blog. We try to do all the right things. We recruit people that are passionate about solving this problem and others that have the best possible supporting skills to solve it.

Furthermore, the issue happened from one mouse click not being made. Is it the engineer's fault? Of course. But why? Why in a security company with smart people who know the risks can this still happen in 2020?

There's only one answer I can think of - the visibility and tools aren't where they need to be.

We are getting back to our roots. The reason we started this company was to 'make sure you aren't blindsighted by an attack you could have prevented.' This situation has doubled my personal resolve to solve this problem."

Chris Ueland

CEO/Co-Founder SecurityTrails

"This last week has reminded me exactly why we decided to build this company. It's made the pain and embarrassment we hope to end painfully close.

Every day there are more and more great services and tools to help people solve their domain problems, but embracing these cutting edge tools and services is a double edged sword. On one hand you have the enhanced capabilities these tools give us, and on the other hand you have the increased surface area that comes with adding another tool to your tool belt.

In the end, we often take a conservative approach to utilizing new services because in all reality - the real risk isn't what we know and do wrong, it's what we don't know and inadvertently do wrong. That worry about not knowing what it is that you don't know has an impact on our ability and confidence to quickly develop.

Engineering teams should be able to quickly assess their overall exposure and understand how their exposure changes as they adopt new tools. If the tools were available and easy to access, we could all afford to take more risks and move quicker, spending more time on solutions and less time on being paranoid about what we missed.

It's our belief that entrepreneurs should be focused on making their vision a reality and not distracted by a fear that a couple of wrong clicks could make all their best practices for nothing. It's this belief, and this passion that drove us to start building SecurityTrails, and our embarrassing lapse last week only reinforced that we must complete our mission."

Courtney Couch

CTO/Co-Founder SecurityTrails

Our commitment to you in the future is to learn from this experience and help others to overcome this unfortunately common problem.

**We will solve this**

We will do everything in our power to solve the problem of unintended attack surface sprawl from technical debt, acquisitions or mistakes.

**We will provide a wide distribution**

We will get tools that provide visibility in the hands of as many people as humanly possible and give teams the knowledge they need to embrace cutting edge tools.

**We will make exposure from an oversight som...