Open in app

Sign in

Write

Sign in

Mastodon

The Automated Phishing Analysis

Sathish Sivaprakash
8 min readMay 18, 2022

--

ThePhish is an automated phishing email analysis tool based on TheHive, Cortex and MISP.

In this article, i will show you how to deploy the automated phishing analysis tool in AWS using Lightsail.

About ThePhish Tool,

  1. Any user can forward the suspicious email to the ThePhish for email analysis.
  2. ThePhish will also create a Ticket in TheHive and use Cortex Analyzer to analysis the email.
  3. Also, it will create the Event in MISP to publish the threat intel.
  4. User gets notification from ThePhish, the analyzed email is safe or malicious.
Source ThePhish (GitHub)

About the Architecture please check the below link,

GitHub - emalderson/ThePhish: ThePhish: an automated phishing email analysis tool

ThePhish is an automated phishing email analysis tool based on TheHive, Cortex and MISP. It is a web application…

github.com

Requirements

AWS — Lightsail Ubuntu 20.4

AWS — Lightsail

Static IP Required

Connect this server via putty

https://lightsail.aws.amazon.com/ls/docs/en_us/articles/lightsail-how-to-set-up-putty-to-connect-using-ssh

Installation,

For installation we can use Portainer Container Management tool to manage the Thephish, Thehive, Cortex and MISP containers and it is easy to deploy and manage.

Install Docker

we need docker to run the docker-compose files, so to install docker please run the following commands.

sudo apt-get updatesudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-commoncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -sudo apt-key fingerprint 0EBFCD88sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"sudo apt-get updatesudo apt-get install docker-ce docker-ce-cli containerd.io docker-composesudo docker versionsudo usermod -aG docker $USER

Install Portainer

This container management tools make your life easier by managing all in single console.

Please execute the below commands to install Portainer.

mkdir -p /opt/portainer && cd /opt/portainercurl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.ymlNano portainer-agent-stack.yml 
Edit the ports like below
ports:
      - "19000:9000"
      - "18000:8000"docker stack deploy --compose-file=portainer-agent-stack.yml portainer

Create the Firewall rule in the LightSail as shown below

Now you can access the portainer <publicIp>:19000

After login — Click Stack and add stack

Portainer

After got the access, please go to stack option and Click add Stack.

Copy the docker-compose below link and paste in the web editor.

ThePhish-TheHive-Cortex-MISP-Docker/docker-compose.yml at main ·…

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below…

github.com

Mention the name of the stack and Deploy it.

Deploy the Docker-Compose File

NOTE : There will be error’s after deployed the stack, but it can be easily resolved. if you want any help leave the comment or reach me.

Access the Consoles

TheHive →<publicip:9000>

Cortex → <publicip:9001>

Thephish →<publicip:8080>

Misp →<https://publicip>

Integration

First enable the bridge connection for integration of each services internally.

Click Containers — and service name

Joining the Bridge Network

Select a network — Bridge Network and Join Network

please refer the below link.

MISP — Thehive Integration

Integrate Misp to Thehive

This is part 8 of the Cortex build. In this part I'm integrating TheHive with MISP and it doesnt go as smooth as I…

blog.agood.cloud

Configure the MISP container

Go to https://publicip and log in with the default credentials:

  • Username: admin@admin.test
  • Password: admin
  1. Create a new organization
  2. Administration -> Add Organization
  3. Name: <YourOrganizationName>
  4. Click on “Generate UUID”
  5. Click on “Submit”
  6. Change settings
  7. Administration -> Server Settings and Maintenance -> MISP Settings
  8. Change the field MISP.live to True
  9. Change the field MISP.baseurl to https://publicip
  10. Change the field MISP.external_base_url to https://publicip
  11. Change the field MISP.org to <YourOrganizationName>
  12. Change the field MISP.host_org_id to <YourOrganizationName>
  13. Create a new user that is used for the integration with TheHive and Cortex
  14. Administration -> Add User
  15. email: sync_user@<YourOrganizationDomain>
  16. organization: <YourOrganizationName>
  17. role: Sync User
  18. Uncheck all the checkboxes
  19. click on “Create user”
  20. Obtain the Authentication key of the Sync User
  21. Administration -> List Users
  22. Click on the “Eye” on the right for the just created user (View)
  23. Click on “Auth Keys”
  24. Delete the already created auth key
  25. Administration -> List Users (again)
  26. Click on the “Eye” on the right for the just created user (again)
  27. Click on “Auth Keys” (again)
  28. Click on “Add authentication key”
  29. Click on “Submit” and save it for later
  30. Enable MISP feeds
  31. Sync Actions -> List Feeds -> Load default feed metadata -> All feeds
  32. Select the feeds to enable
  33. Click on “Enable selected”

Configure the Cortex container

  1. Go to http://publicIp:9001 and click on "Update database"
  2. Create a new admin user
  3. Login: admin@<YourOrganizationName>
  4. Name: admin
  5. Password: <Password>
  6. Create a new organization
  7. Organizations -> Add Organization
  8. Name: <YourOrganizationName>
  9. Description: <YourOrganizationDescription>
  10. Create a new orgadmin user in that organization
  11. Click on the newly created organization <YourOrganizationName>
  12. Click on “Add user”
  13. Login: thephish@<YourOrganizationName>
  14. Full name: ThePhish
  15. Roles: read, analyze, orgadmin
  16. Click on “New password” for the newly created user and set a password for that user
  17. Create another user in that organization that is used for the integration with TheHive and to use the API
  18. Click on the newly created organization <YourOrganizationName>
  19. Click on “Add user”
  20. Login: integration_account@<YourOrganizationName>
  21. Full name: integration_account
  22. Roles: read, analyze
  23. Click on “Create API key” and then on “Reveal” for the newly created user and save it for later
  24. Log out the admin user and log in the orgadmin user (ThePhish)

Enable the Mailer responder

On Cortex <https://publicip:9001>, while logged in with the orgadmin user, do the following:

  1. Organization -> Responders
  2. Enable the Mailer responder
  3. Configure the Mailer responder
  4. from: <YourGmailEmailAddress>
  5. smtp_host :smtp.gmail.com
  6. smtp_port: 587
  7. smtp_user: <YourGmailEmailAddress>
  8. smtp_pwd: <YourGmailEmailAddressAppPassword>

Integrate Cortex with MISP

You should see the IP address that has been assigned in the default bridge network to the MISP container. It will be used to configure the MISP_2_1 analyzer on Cortex.

Now, on Cortex, while logged in with the orgadmin user, do the following:

  1. Organization -> Analyzers
  2. Enable the MISP_2_1 analyzer
  3. Configure the MISP_2_1 analyzer
  4. url: https://<IPAddressOfTheMISPInstanceInTheDefaultBridgeNetwork>
  5. key: <AutheticationKeyOfTheSyncUserCreatedOnMISP>
  6. cert_check: False

Enable the analyzers

Abuse_Finder_3_0
Urlscan_io_Search_0_1_1
DShield_lookup_1_0
CyberCrime-Tracker_1_0
Cyberprotect_ThreatScore_1_0
MISP_2_1
URLhaus_2_0

Integrate TheHive with Cortex

https://blog.agood.cloud/posts/2019/09/27/integrate-thehive-and-cortex/

Edit the cortex part of the file thehive/application.conf to replace the XXXXXXXXXXXXXXx with the API key of the integration_account created in Cortex.

play.modules.enabled += org.thp.thehive.connector.cortex.CortexModule
cortex {
  servers = [
    {
      name = local
      url = "http://cortex:9001"
      auth {
        type = "bearer"
        key = "XXXXXXXXXXXXXXx"
      }
      # HTTP client configuration (SSL and proxy)
      #  wsConfig {}
     # List TheHive organisation which can use this Cortex server. All ("*") by default
     # includedTheHiveOrganisations = ["*"]
     # List TheHive organisation which cannot use this Cortex server. None by default
     # excludedTheHiveOrganisations = []
    }
  ]
  # Check job update time intervalcortex
  refreshDelay = 5 seconds
  # Maximum number of successive errors before give up
  maxRetryOnError = 3
  # Check remote Cortex status time interval
  statusCheckInterval = 1 minute
}

Integrate TheHive with MISP

Edit the misp part of the file thehive/application.conf to replace the XXXXXXXXXXXXXXx with the Authentication key of the sync_user created in MISP.

# MISP configuration
play.modules.enabled += org.thp.thehive.connector.misp.MispModule
misp {
  interval: 5 min
  servers: [
    {
      name = "MISP THP"            # MISP name
      url = "https://misp/" # URL or MISP
      auth {
        type = key
        key = "XXXXXXXXXXXXXXx"             # MISP API key
      }
      wsConfig { ssl { loose { acceptAnyCertificate: true } } }
    }
  ]
}

Configure the TheHive container

  1. Go to http://localhost:9000 and log in with the default credentials:
  • Username: admin@thehive.local
  • Password: secret
  1. Create a new organization
  2. Click on “New organization”
  3. Name: <YourOrganizationName>
  4. Description: <YourOrganizationDescription>
  5. Create a new orgadmin user in that organization
  6. Click on the newly created organization <YourOrganizationName>
  7. Click on “Create new user”
  8. Login: thephish@<YourOrganizationName>
  9. Full name: ThePhish
  10. Profile: org-admin
  11. Click on “New password” for the newly created user and set a password for that user
  12. Click on “Create API key” and then on “Reveal” for the newly created user and save it for later
  13. Log out the admin user and log in the orgadmin user (ThePhish)

Configure the ThePhish container

The file configuration.json is the global configuration file that allows setting the parameters for the connection to the mailbox and to the instances of TheHive, Cortex and MISP. It also allows setting parameters related to the cases that will be created on TheHive.

{
	"imap" : {
		"host" : "imap.gmail.com",
		"port" : "993",
		"user" : "",
		"password" : "",
		"folder" : "inbox"
	},
	"thehive" : {
		"url" : "http://thehive:9000",
		"apikey" : ""
	},
	"cortex" : {
		"url" : "http://cortex:9001",
		"apikey" : "",
		"id" : "local"
	},
	"misp" : {
		"id" : "MISP THP"
	},
	"case" : {
		"tlp" : "2",
		"pap" : "2",
		"tags" : ["email", "ThePhish"]
	}
}
  • In the imap part, if you are using a Gmail address, you only need to set the username used to connect to the IMAP server (which is your email address) and the app password.
  • In the thehive part you have to set the URL at which the TheHive instance is reachable and set the API key of the user created on TheHive that ThePhish will use to interact with TheHive.
  • In the cortex part you have to set the URL at which the Cortex instance is reachable and set the API key of the user created on Cortex that both ThePhish and TheHive will use to interact with Cortex. Moreover, you have to set the ID given to the Cortex instance.
  • In the misp part you only have to set the ID given to the MISP instance.
  • In the case part you can set the default TLP and PAP levels for the cases created by ThePhish and also the tags that will be applied to them at their creation.

SOURCE :

https://github.com/emalderson/ThePhish/tree/master/docker

https://github.com/emalderson/ThePhish#configure-the-analyzers

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -

USE CASE

  1. Send a Email using forward as attachment to ThePhish analysis Email.

2. Go to ThePhish Console <publicip:8080>

3. Click List Email and it will the show the emails for analysis.

NOTE : It only read emails if you sent using “Forward as Attachment”

4. Click Analyze → to start the analysis

Final Result

6. You will also receive the acknowledgement & results in email.

Cyber
Cybersecurity
Information Security
Phishing
Phishing Awareness

--

--

Sathish Sivaprakash
Sathish Sivaprakash

Written by Sathish Sivaprakash

85 Followers
32 Following

Threat Intelligence Analyst and Security Automation | Located in Abu Dhabi, UAE.

Responses (1)