By Tony Lee
Does your organization have a threat intel chatbot to do your bidding? If not, you should give CyBot (
https://github.com/cylance/CyBot) a try. It is designed not only for threat intel groups, but also incident responders, and SOCs. CyBot was born in HipChat, then migrated to Slack after the merger, and now finally adapted to work for Microsoft Teams. Don't get me wrong, this article is not an endorsement for using Microsoft Teams, but you don't always get to pick your chat platform--especially when Microsoft has the ability to bundle Teams with o365 subscriptions. The upside is that the Teams platform does seem to be improving over time as its market share grows (largely a result of the previously mentioned bundling). In fact, in July of 2019, Microsoft announced that they surpassed Slack in number of daily users. Due to these factors, with the herculean help of Igor Vasilcovsky, we looked into getting CyBot’s underlying Errbot platform onto Microsoft Teams. Warning: Due to Microsoft’s rapid (and sometimes frustrating) development efforts, this integration could break or change at a moment’s notice—caveat emptor. 😐
https://www.theverge.com/2019/7/11/20689143/microsoft-teams-active-daily-users-stats-slack-competition
Figure 1: CyBot Working in Microsoft Teams
Article Outline
- Our Setup Assumptions
- Install and initialize Errbot
- Install and Configure nginx Reverse Proxy
- Generate and Install a Valid SSL Certificate
- Create the Microsoft Bot Channel
- Download and Configure the Botframework Backend
- Modifying the botframework plugin
- Preliminary Test
- Add your Bot to Microsoft Teams
- Package the Bot
- Use Microsoft Teams App Studio
Our Setup Assumptions
- Hosting Errbot in AWS on Ubuntu 18.04
- Using Microsoft’s “Bot Channels Registration” to link Errbot to Teams
- Even though documentation states that Bot Channels Registration is free, we could not get it to work with a free subscription. It appears to cost 50 cents per 1,000 messages.
- Domain name purchased VIA AWS Route 53 (reasonably priced)
- nginx reverse proxy
- Free (but legitimate) SSL certificate provisioned via letsencrypt.org Certbot
- Errbot installation path used in our examples: /home/ubuntu/errbot-root/
- If using the Bot Framework Emulator for testing/troubleshooting, you may need to sign up for a free ngrok account to tunnel traffic remotely
Pro-tip: Use the screen command to run errbot in since it continues to run even when you are disconnected from the terminal.
Known Limitations
Solved
- Teams does not use full markdown and documentation is incorrect
Not solved
- Messages must be direct messages (not group chat)
- Copy and paste from Teams adds extra CR LF characters which break the errbot recognition
- Admin restrictions and private plugins need to be handled
- Teams mobile app ignores \r\n formatting (web and desktop client do not)
Traffic Flow
The below diagram shows the resulting traffic flow from Microsoft to our errbot server.
Figure 2: Flow of traffic
Install and initialize Errbot
After setting up the Ubuntu 18.04 EC2 instnce and purchasing our domain, we installed Errbot using the following:
sudo apt update
sudo apt install python3-dev libssl-dev libffi-dev
sudo apt install python3-pip
sudo pip3 install errbot
mkdir ~/errbot-root
cd ~/errbot-root
errbot --init
After the initialization, we need to test errbot and load the errbot Webserver plugin:
errbot
>>> !plugin config Webserver
>>> !webstatus
>>> !webhook test /echo
Note: If the above webhook test did not work (result in a 200), try the following:
>>> !plugin config Webserver {'HOST': '0.0.0.0', 'PORT': 3141, 'SSL': {'certificate': '', 'enabled': False, 'host': '0.0.0.0', 'key': '', 'port': 3142}}
>>> !shutdown --confirm
Note: Before continuing with the next section, have your domain name purchased and forwarding traffic to your server. We used AWS Route 53 for our domain name purchase since we were also hosting the instance in AWS, but feel free to use any service that allows you to direct or transparently forward the domain to your host. Note: Godaddy’s domain forwarding did not work with our SSL certificate.
Install and Configure nginx Reverse Proxy
Interestingly enough, errbot cannot directly receive HTTPS communication from Microsoft Bot Channels for some reason so a reverse proxy is required. For this task, we will use nginx due to its lightweight and flexible nature. To install and initialize, we used the following:
sudo apt install nginx
sudo nginx -s reload
Test for port 80 being open:
netstat -plant | grep :80
Note: You may need to temporarily open TCP 80 and 443 on the AWS firewall to test nginx and register for the SSL certificate.
To ensure external access, use a browser to navigate to the server’s port 80 to see the default nginx page.
Determine the default nginx install location using the following command:
sudo nginx -t
ubuntu@ip-xxx-xx-xx-xx:~$
sudo nginx -t
nginx: the configuration file
/etc/nginx/nginx.conf syntax is ok
nginx: configuration file
/etc/nginx/nginx.conf test is successful
|
Now that we know the default location (for us it was /etc/nginx/nginx.conf), let’s create our test port and redirection to errbot:
sudo vim /etc/nginx/conf.d/cybot.conf
server {
listen 3143;
server_name www.yourdomain.com;
location / {
proxy_pass http://localhost:3141/;
}
}
|
After creating this test port and opening the Amazon firewall for port 3143, restart nginx and use a web browser to navigate to http://www.yourdomain.com:3143 to test the unencrypted nginx forwarding on to the errbot Web Server listener.
sudo service nginx restart
Figure 3: This is an expected message and means the redirection from nginx to the errbot webserver is working
Generate and Install a Valid SSL Certificate
WARNING: Microsoft’s Bot Channels Registration, Azure portal Web chat, and Bot emulator do not work with self-signed certs—
thus you need a legitimate SSL certificate. We will create a free, but legitimate letsencrypt.org certificate using certbot. Please remember to have your domain name purchased and ready to assign to this certificate, plus port 80 and 443 on your AWS firewall open to this host.
Prep for Certbot
sudo apt-get update
sudo apt-get install software-properties-common
sudo add-apt-repository universe
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
Install Certbot
sudo apt-get install certbot python-certbot-nginx
Use Certbot to generate and configure nginx
Note: When asked for the CN, make sure you use the Fully Qualified Domain Name (FQDN) of your server. Ex: www.yourdomain.com
sudo certbot --nginx
Hopefully creating the certificate was successful and we can modify our config file to include it.
sudo vim /etc/nginx/conf.d/cybot.conf
server {
#listen 3143;
listen 3142 ssl;
ssl_certificate /etc/letsencrypt/live/<yoursite>/fullchain.pem;
# managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/<yoursite>/privkey.pem;
# managed by Certbot
location / {
proxy_pass http://localhost:3141/;
}
}
|
The above config commented our port 3143 which we use as an unencrypted test port. 3142 is the SSL receiver for the Microsoft communications which then forwards the traffic on to localhost 3141. If for whatever reason, you need to test the functionality without the complexity of encryption (i.e. using Bot Framework Emulator), feel free to uncomment the test port 3143 and restart nginx.
Create the Microsoft Bot Channel
Warning: Microsoft seems to indicate that this integration is free, but we could not get this integration working with the free trial. After the free trial was over, we had to select a paid tier and it worked. We found this to be annoying.
Source:
https://azure.microsoft.com/en-us/pricing/details/bot-service/
Create the Microsoft Bot Channel
- Go to Azure Portal - https://portal.azure.com/
- Go to "Create a Resource"
- Search the Marketplace > "Bot Channel" > Bot Channels Registration > Create
- Fill the form and wait for the resource to be created
- Note for Messaging endpoint:
- https://<yoursite>:3142/botframework
Figure 4: Creating the Bot Channels Registration
Obtain appId and appPassword for errbot integration
When finished with Bot Channels creation, save off the application id (Dashboard -> Your Bot name -> Settings -> MSFT App ID)
ex: 7635snip-snip-snip-snip-snip4cf15a0f
Obtain and save the appPassword as well (Dashboard -> Your Bot name -> Settings -> Manage (next to MSFT App ID) -> New Client Secret
ex: 9@snipsnipsnipsnipsnipsnipCg6F
Figure 5: Obtaining the appId and appPassword
Download and Configure the Botframework Backend
First of all… A huge thanks goes out to Igor Vasilcovsky for creating and helping troubleshoot this botframework backend. This thanks cannot be understated. Now that we have the Bot Channels Registration appId and appPassword, on the AWS Ubuntu terminal run the following commands:
cd ~/errbot-root
git clone https://github.com/vasilcovsky/errbot-backend-botframework.git
cp config.py config.orig
vim config.py
import logging
#BACKEND = 'Text'
BACKEND = 'BotFramework' # Errbot will start in text mode (console
only mode) and will answer commands from there.
BOT_DATA_DIR = r'/path/to/errbot-root/data'
BOT_EXTRA_PLUGIN_DIR = r'/path/to/errbot-root/plugins'
BOT_EXTRA_BACKEND_DIR = r'/path/to/errbot-root/errbot-backend-botframework/'
BOT_LOG_FILE = r'/path/to/errbot-root/errbot.log'
BOT_LOG_LEVEL = logging.DEBUG
BOT_IDENTITY = {
'appId': '<ENTER IT HERE>',
'appPassword': '<ENTER IT HERE>'
}
BOT_ADMINS = ('@CHANGE_ME',
) # !! Don't leave that to
"@CHANGE_ME" if you connect your errbot to a chat system !!
|
Note: Our /path/to/ was /home/ubuntu/
Example values for appId and appPassword (from above):
BOT_IDENTITY = {
'appId': ‘7635snip-snip-snip-snip-snip4cf15a0f’,
'appPassword': ‘9@snipsnipsnipsnipsnipsnipCg6F’
}
Modify the botframework plugin
To make the output prettier in teams we modified Igor’s botframework plugin. This modification detects the number of lines of output and then adds either adds an in-line or multi-line code block around the output. Lines 156 – 159 were added to the following file
vim ~/errbot-root/errbot-backend-botframework/botframework.py
154 def _build_reply(self, msg):
155 conversation =
msg.extras['conversation']
156 if
len(msg.body.splitlines())==1:
157
msg.body="`" + msg.body + "`"
158 else:
159 msg.body="```" + msg.body
160
161 payload = {
|
Now that the config file is complete and pointing at the modified Botframework backend, start/restart errbot
errbot
Preliminary Test
There are a couple of ways to test the receiving end (nginx, errbot, etc)
1) Via a Web Browser
With the errbot window open, you can test the nginx redirection using a simple web browser, navigating to your messaging endpoint:
https://<yoursite>:3142/botframework
If you see traffic arrive in errbot, it means your domain, SSL cert, nginx redirection, and errbot Web Server plugin are all working. If this does not work, you may need to check firewall settings and possibly configurations for everything.
Figure 6: Browser was able to corectly route from SSL nginx to errbot Webserver plugin
2) Test via the Web Chat
Now that you tested the basic connectivity, let’s test the Microsoft side of the house by using the Web Chat feature within the Azure portal.
1.
Go to Azure Portal -
https://portal.azure.com/
2.
Navigate to: Dashboard -> Your Bot name -> Test in Web Chat
If everything seems ok, try to run !help. If all is well, you should receive the help menu as shown in the screenshot below.
Figure 7: Errbot help menu exercised through the Azure Portal Web Chat
If you did not see the help menu, there is something wrong with the Microsoft piece of the integration and you may need to use the Botnet Framework Emulator to troubleshoot (see that section for more details).
If you see the help menu, you need to now add the MSTeams channel to your bot and then communicate with the bot through Teams.
Add your Bot to Microsoft Teams
Now that the webchat worked, the next step is to add the MSTeams channel to your bot.¬ Once added you should be able to communicate with the Bot over the via Teams. The three way we will outline are the following:
1)
Use the appId for direct communication <-- Worked on our Teams instance, but not others
2)
Publish publicly to Teams <-- We are avoiding this..
3)
Package the bot to import into a third part Teams instance <-- Worked for other Teams
Figure 8: Adding MSTeams channel to the bot
Chat with bot via the appId
One of the trickier steps in the process is figuring out how to actually communicate with the Bot. The easiest way is to use the appId that we noted earlier. Example:
7635snip-snip-snip-snip-snip4cf15a0f
Figure 9: Finding the bot in teams via the appId.
This seems to work for direct messaging the bot, but it does not seem possible to add the bot to a team via the appId. To accomplish this, we have to package the bot so we could upload it to our organization and use privately. The alternative is to publish the bot to Teams at large and allow anyone to use it, but the risk of misuse greatly increases.
Package the Bot
To upload the bot to Teams, we must package it up first. A Teams apps package is relatively simple and consists of a JSON manifest file and two icons.
Figure 10: Teams apps package
Icon guidance
The icons must conform to a certain size and filename. For the latest guidance please consult the following link:
https://docs.microsoft.com/en-us/microsoftteams/platform/concepts/apps/apps-package
Manifest guidance
Even though the Azure portal seems to contain a manifest file, strangely enough it is not the same format as what is required for app upload into Teams—Go Microsoft! The easiest way to generate the required manifest is to use App Studio within Teams. Please see the link below on how to activate it within your Teams instance.
https://docs.microsoft.com/en-us/microsoftteams/platform/get-started/get-started-app-studio
Use Microsoft Teams App Studio
To create the bot, open App Studio within Microsoft Teams
1)
Click Manifest editor
2)
Create a new app
3)
Details section: Fill in details shown below. (The app ID is the same app ID from previous steps)
Figure 11: Creating the Teams Package
4)
Capabilities Section: Bots subsection – Fill in the details below:
Figure 12: Capabilities -> Bot Section
Install or Download the app to publish later
Once the at the end of the App Studio creation process (Finish -> Test and distribute), you can immediate test the app by using the Install button or you can download the app. Provided that you have permissions to sideload the app, you can try the install button.
Figure 13: Options for Install or Download
If installing on a different Teams instance, you may be able to sideload the app with a manual upload using Apps -> Upload a custom app. Then just point to the app zip file.
Figure 14: Sideload the app if this option is enabled
If that is disabled, you may need to have your Azure admins install the app for you. Once installed, you will be able to add your bot to chats just like a person.
https://docs.microsoft.com/en-us/microsoftteams/tenant-apps-catalog-teams
Install CyBot Plugins
Now that the heavy lifting is done, it is time to install the CyBot plugins.
cd /tmp
git clone https://github.com/cylance/CyBot.git
cp -a CyBot/plugins/* ~/errbot-root/plugins/
Restart errbot and use !commands to start using the CyBot plugins.
Figure 15: Chatting with CyBot in Teams
Added Security
Now that everything is working, let's add some additional security by only allowing traffic on port 3142 from the following ranges:
- 13.104.0.0/14
- 13.96.0.0/13
- 13.64.0.0/11
Troubleshooting
If you run into any issues and need help, please see our next article on Troubleshooting CyBot on Microsoft Teams:
http://securitysynapse.blogspot.com/2019/09/troubleshooting-cybot-on-microsoft-teams.html
Conclusion
If you are still with us… you are a hero. Microsoft did not design the “bring your own bot” to be simple by any means, however it seems to work for now. We spent the past two weeks cleaning up plugin output to be more Teams friendly while still maintaining compatibility with Slack and other platforms. We will continue to improve the integration and plugin functionality over time, but feel free to let us know if you have any issues.
Special Thanks
Special thanks goes to the errbot team for making this flexible cross-chat platform available. Special thanks also goes to Igor Vasilcovsky for his work on the botframework backend.
