The Automated Phishing Analysis
ThePhish is an automated phishing email analysis tool based on TheHive, Cortex and MISP.
In this article, i will show you how to deploy the automated phishing analysis tool in AWS using Lightsail.
About ThePhish Tool,
- Any user can forward the suspicious email to the ThePhish for email analysis.
- ThePhish will also create a Ticket in TheHive and use Cortex Analyzer to analysis the email.
- Also, it will create the Event in MISP to publish the threat intel.
- User gets notification from ThePhish, the analyzed email is safe or malicious.
About the Architecture please check the below link,
Requirements
AWS — Lightsail Ubuntu 20.4
Static IP Required
Connect this server via putty
Installation,
For installation we can use Portainer Container Management tool to manage the Thephish, Thehive, Cortex and MISP containers and it is easy to deploy and manage.
Install Docker
we need docker to run the docker-compose files, so to install docker please run the following commands.
sudo apt-get updatesudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-commoncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -sudo apt-key fingerprint 0EBFCD88sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"sudo apt-get updatesudo apt-get install docker-ce docker-ce-cli containerd.io docker-composesudo docker versionsudo usermod -aG docker $USER
Install Portainer
This container management tools make your life easier by managing all in single console.
Please execute the below commands to install Portainer.
mkdir -p /opt/portainer && cd /opt/portainercurl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.ymlNano portainer-agent-stack.yml
Edit the ports like below
ports:
- "19000:9000"
- "18000:8000"docker stack deploy --compose-file=portainer-agent-stack.yml portainer
Create the Firewall rule in the LightSail as shown below
Now you can access the portainer <publicIp>:19000
After login — Click Stack and add stack
After got the access, please go to stack option and Click add Stack.
Copy the docker-compose below link and paste in the web editor.
Mention the name of the stack and Deploy it.
NOTE : There will be error’s after deployed the stack, but it can be easily resolved. if you want any help leave the comment or reach me.
Access the Consoles
TheHive →<publicip:9000>
Cortex → <publicip:9001>
Thephish →<publicip:8080>
Misp →<https://publicip>
Integration
First enable the bridge connection for integration of each services internally.
Click Containers — and service name
Select a network — Bridge Network and Join Network
please refer the below link.
MISP — Thehive Integration
Configure the MISP container
Go to https://publicip
and log in with the default credentials:
- Username:
admin@admin.test
- Password:
admin
- Create a new organization
- Administration -> Add Organization
- Name:
<YourOrganizationName>
- Click on “Generate UUID”
- Click on “Submit”
- Change settings
- Administration -> Server Settings and Maintenance -> MISP Settings
- Change the field
MISP.live
toTrue
- Change the field
MISP.baseurl
tohttps://publicip
- Change the field
MISP.external_base_url
tohttps://publicip
- Change the field
MISP.org
to<YourOrganizationName>
- Change the field
MISP.host_org_id
to<YourOrganizationName>
- Create a new user that is used for the integration with TheHive and Cortex
- Administration -> Add User
- email:
sync_user@<YourOrganizationDomain>
- organization:
<YourOrganizationName>
- role:
Sync User
- Uncheck all the checkboxes
- click on “Create user”
- Obtain the Authentication key of the
Sync User
- Administration -> List Users
- Click on the “Eye” on the right for the just created user (View)
- Click on “Auth Keys”
- Delete the already created auth key
- Administration -> List Users (again)
- Click on the “Eye” on the right for the just created user (again)
- Click on “Auth Keys” (again)
- Click on “Add authentication key”
- Click on “Submit” and save it for later
- Enable MISP feeds
- Sync Actions -> List Feeds -> Load default feed metadata -> All feeds
- Select the feeds to enable
- Click on “Enable selected”
Configure the Cortex container
- Go to
http://publicIp:9001
and click on "Update database" - Create a new admin user
- Login:
admin@<YourOrganizationName>
- Name:
admin
- Password:
<Password>
- Create a new organization
- Organizations -> Add Organization
- Name:
<YourOrganizationName>
- Description:
<YourOrganizationDescription>
- Create a new orgadmin user in that organization
- Click on the newly created organization
<YourOrganizationName>
- Click on “Add user”
- Login:
thephish@<YourOrganizationName>
- Full name:
ThePhish
- Roles:
read, analyze, orgadmin
- Click on “New password” for the newly created user and set a password for that user
- Create another user in that organization that is used for the integration with TheHive and to use the API
- Click on the newly created organization
<YourOrganizationName>
- Click on “Add user”
- Login:
integration_account@<YourOrganizationName>
- Full name:
integration_account
- Roles:
read, analyze
- Click on “Create API key” and then on “Reveal” for the newly created user and save it for later
- Log out the admin user and log in the orgadmin user (ThePhish)
Enable the Mailer responder
On Cortex <https://publicip:9001>, while logged in with the orgadmin user, do the following:
- Organization -> Responders
- Enable the Mailer responder
- Configure the Mailer responder
- from:
<YourGmailEmailAddress>
- smtp_host :
smtp.gmail.com
- smtp_port:
587
- smtp_user:
<YourGmailEmailAddress>
- smtp_pwd:
<YourGmailEmailAddressAppPassword>
Integrate Cortex with MISP
You should see the IP address that has been assigned in the default bridge network to the MISP container. It will be used to configure the MISP_2_1 analyzer on Cortex.
Now, on Cortex, while logged in with the orgadmin user, do the following:
- Organization -> Analyzers
- Enable the MISP_2_1 analyzer
- Configure the MISP_2_1 analyzer
- url:
https://<IPAddressOfTheMISPInstanceInTheDefaultBridgeNetwork>
- key:
<AutheticationKeyOfTheSyncUserCreatedOnMISP>
- cert_check:
False
Enable the analyzers
Abuse_Finder_3_0
Urlscan_io_Search_0_1_1
DShield_lookup_1_0
CyberCrime-Tracker_1_0
Cyberprotect_ThreatScore_1_0
MISP_2_1
URLhaus_2_0
Integrate TheHive with Cortex
https://blog.agood.cloud/posts/2019/09/27/integrate-thehive-and-cortex/
Edit the cortex part of the file thehive/application.conf
to replace the XXXXXXXXXXXXXXx
with the API key of the integration_account
created in Cortex.
play.modules.enabled += org.thp.thehive.connector.cortex.CortexModule
cortex {
servers = [
{
name = local
url = "http://cortex:9001"
auth {
type = "bearer"
key = "XXXXXXXXXXXXXXx"
}
# HTTP client configuration (SSL and proxy)
# wsConfig {}
# List TheHive organisation which can use this Cortex server. All ("*") by default
# includedTheHiveOrganisations = ["*"]
# List TheHive organisation which cannot use this Cortex server. None by default
# excludedTheHiveOrganisations = []
}
]
# Check job update time intervalcortex
refreshDelay = 5 seconds
# Maximum number of successive errors before give up
maxRetryOnError = 3
# Check remote Cortex status time interval
statusCheckInterval = 1 minute
}
Integrate TheHive with MISP
Edit the misp part of the file thehive/application.conf
to replace the XXXXXXXXXXXXXXx
with the Authentication key of the sync_user
created in MISP.
# MISP configuration
play.modules.enabled += org.thp.thehive.connector.misp.MispModule
misp {
interval: 5 min
servers: [
{
name = "MISP THP" # MISP name
url = "https://misp/" # URL or MISP
auth {
type = key
key = "XXXXXXXXXXXXXXx" # MISP API key
}
wsConfig { ssl { loose { acceptAnyCertificate: true } } }
}
]
}
Configure the TheHive container
- Go to
http://localhost:9000
and log in with the default credentials:
- Username:
admin@thehive.local
- Password:
secret
- Create a new organization
- Click on “New organization”
- Name:
<YourOrganizationName>
- Description:
<YourOrganizationDescription>
- Create a new orgadmin user in that organization
- Click on the newly created organization
<YourOrganizationName>
- Click on “Create new user”
- Login:
thephish@<YourOrganizationName>
- Full name:
ThePhish
- Profile:
org-admin
- Click on “New password” for the newly created user and set a password for that user
- Click on “Create API key” and then on “Reveal” for the newly created user and save it for later
- Log out the admin user and log in the orgadmin user (ThePhish)
Configure the ThePhish container
The file configuration.json
is the global configuration file that allows setting the parameters for the connection to the mailbox and to the instances of TheHive, Cortex and MISP. It also allows setting parameters related to the cases that will be created on TheHive.
{
"imap" : {
"host" : "imap.gmail.com",
"port" : "993",
"user" : "",
"password" : "",
"folder" : "inbox"
},
"thehive" : {
"url" : "http://thehive:9000",
"apikey" : ""
},
"cortex" : {
"url" : "http://cortex:9001",
"apikey" : "",
"id" : "local"
},
"misp" : {
"id" : "MISP THP"
},
"case" : {
"tlp" : "2",
"pap" : "2",
"tags" : ["email", "ThePhish"]
}
}
- In the imap part, if you are using a Gmail address, you only need to set the username used to connect to the IMAP server (which is your email address) and the app password.
- In the thehive part you have to set the URL at which the TheHive instance is reachable and set the API key of the user created on TheHive that ThePhish will use to interact with TheHive.
- In the cortex part you have to set the URL at which the Cortex instance is reachable and set the API key of the user created on Cortex that both ThePhish and TheHive will use to interact with Cortex. Moreover, you have to set the ID given to the Cortex instance.
- In the misp part you only have to set the ID given to the MISP instance.
- In the case part you can set the default TLP and PAP levels for the cases created by ThePhish and also the tags that will be applied to them at their creation.
SOURCE :
https://github.com/emalderson/ThePhish/tree/master/docker
https://github.com/emalderson/ThePhish#configure-the-analyzers
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -
USE CASE
- Send a Email using forward as attachment to ThePhish analysis Email.
2. Go to ThePhish Console <publicip:8080>
3. Click List Email and it will the show the emails for analysis.
NOTE : It only read emails if you sent using “Forward as Attachment”
4. Click Analyze → to start the analysis
6. You will also receive the acknowledgement & results in email.