Exercise 1.3: Proactive Bot Defense¶

Create Policy¶

1. Open the Terminal application.

2. Run the following curl command to verify the site is loading without issue from this command line http utility. If the curl command is not successful (you are getting a “request rejected” error page), please let an instructor know.

   curl https://webgoat.f5demo.com/WebGoat/login -k -v | more

Input

Output

1. On the Main tab, click Security > DoS Protection > DoS Profiles. The DoS Profiles screen opens.

   

2. Click on the Create button.

3. Name the policy webgoat_DoS and click Finished to complete the creation of this DoS profile.

   

Configure Policy¶

1. Click the newly created webgoat_DoS profile listed under the Security > Dos Protection > DoS Profiles list.

2. The profile’s properties menu will be displayed initially. Click on the Application Security tab at the top of this menu to begin configuring the policy.

   

3. Under the Application Security tab > General Settings click the Edit link on the right-hand side of General Settings box and then check the Enabled check box for Application Security to enable the DoS profile and allow additional settings to be configured.

   

4. Select Proactive Bot Defense under the list of Application Security options for this DoS profile.

5. Click the Edit link on the right for the Application Security > Proactive Bot Defense menu and select Always from the drop-down menu for Operation Mode.

6. Set the Grace Period to 20 seconds. We will observe this in action shortly.

   

7. Notice that for Block requests from suspicious browsers the Block Suspicious Browsers setting is enabled by default.

8. At this point, you may want to take a moment and explore the other defaults that were turned on such as TPS based detection and BOT Signatures. Please don’t modify the defaults.

9. Click the Update button to complete the Proactive Bot Defense webgoat_DoS profile.

   

Apply Proactive Bot Defense Policy¶

1. Under Local Traffic > Virtual Servers, click on webgoat.f5demo.com_https_vs.

2. Click on Policies under the Security tab at the top of the webgoat.f5demo.com_https_vs details menu.

3. In the DoS Protection Profile drop down menu, select Enabled... and then select the webgoat_DoS for the profile.

4. Click on the Update button to apply the policy.

   

Create Bot Defense Logging Profile¶

1. Open a new tab for the Configuration Utility and navigate to:  Security > Event Logs > Logging Profiles then click the plus icon.

2. Enter a Profile Name bot-defense, select the checkbox for Bot Defense.

3. Under the Bot Defense logging section, select the checkboxes for the following: Local Publisher, Log Illegal Requests, Log Bot Signature Matched Requests and Log Challenged Requests.

4. Click Finished.

   Note

   You could have also modified the existing waf_allrequests logging profile and added BOT logging definitions.

   

Apply Bot Defense Logging Profile¶

1. Under Local Traffic > Virtual Servers, click on webgoat.f5demo.com_https_vs.

2. Click on Policies under the Security tab at the top

3. Within the Available logging profiles menu, select bot-defense and then click the << arrows to move the logging policy to the Selected profile.

4. Click on the Update button to apply the policy.

   Note

   You can associate multiple logging profiles with a given virtual server. F5 allows for an incredible amount of logging flexibility. Most commonly you would have DoS, Bot Defense and ASM Security Policy events logged to a centralized SIEM platform, but there may be additional logging requirements such as a web team that would be interested in Bot Defense logs solely, while the SIEM continues to receive the union of DoS, Bot Defense and ASM Security Policy events.

   

Test the Proactive Bot Defense Policy¶

1. From the command line execute the following command several times:

   curl https://webgoat.f5demo.com/WebGoat/login -k -v | more

2. Once the Grace Period of 20 seconds has expired you will see ASM start escalating the defense and start to return a javascript challenge.

This bot is getting shot down in flames!

Validate that the Proactive Bot Defense Policy is Working¶

1. Navigate to Security > Event Logs > Bot Defense > Requests.
2. Notice that the detected bot activity has been logged and is now being displayed for review.

1. Note the stated reason for the request being blocked. You may have to scroll to the right to see this reason. What was the stated reason?

BOT Signatures¶

1. Navigate to Security > DoS Protection > DoS Profiles

2. Click on the webgoat_DoS profile and then the Application Security tab to configure the policy.

3. Select Proactive Bot Defense under the list of Application Security options.

4. In the Application Security > Proactive Bot Defense section, click the Edit link for Operation Mode and then change the setting from Always to During Attack and click Update to complete the policy change.

   

5. Run cURL again: curl https://webgoat.f5demo.com/WebGoat/login -k -v | more

cURL is considered an HTTP Library tool and falls in the Benign Category.

Selectively Blocking BOT Categories¶

1. Under your webgoat_DoS profile in Application Security > Bot Signatures click on the Edit link for the Bot Signature Categories section.

   

2. Change the HTTP Library action from None to Block under the Benign Categories section and click Update to apply the policy changes.

   

3. Run cURL again: curl  https://webgoat.f5demo.com/WebGoat/login -k -v | more

   

   Whammo!!!… as soon as the BOT is revealed… the connection is dropped. The TLS doesn’t get established.

   Let’s say we actually DO want to allow cURL or another automated tool. We may have developers that rely on curl so let’s whitelist just that.

To Whitelist cURL:

1. Edit the Bot Signatures list and find curl. Move it to disabled signatures and click Update.

1. Run cURL again: curl https://webgoat.f5demo.com/WebGoat/login -k -v | more and you should be back in business. By now you should know the expected output.
2. Change HTTP Library to: Report and remove CURL from the whitelist.

1. Modify the webgoat_DOS Dos Profile operation Operation Mode to: Always and click Update.

cURL from Different Geolocations¶

1. Open Local Traffic > Virtual Servers and click on webgoat.f5demo.com_https_overlay_vs. Go to the Resources horizontal tab and verify that the iRule webgoat_overlay is applied. Freel free to check out the code in the iRule. This code and BIG-IP flexibility makes lab testing and simulations a breeze.

2. Modify the cURL command to point at the overlay virtual server and run several times: curl https://10.1.10.146/WebGoat/login -k -v | more
3. Review the event logs at Event Logs > Bot Defense You will now see geo-data for the BOT connection attempts.

4. Navigate to Security > Overview > Application > Traffic and review the default report elements. You can change the widget time frames to see more historical data.
5. Click Overview > Application > Traffic and override the timeframe to past year:

6. Take some time reviewing this screen and practice adding a new widget to see additional reporting elements:
7. Click the DoS tab at the top. In some time…The DOS Visibility Screen loads.

Although there have not been any L7 DoS attacks some of the widgets along the right contain statistics from the BOT mitigations. Change the time window (top left) from 5 minutes to “All Time” so see more data.

8. Click the Analysis tab at the top and review the graphs available to you.

9. Click the Custom Page tab at the top and review the graphs available to you.

Please feel free to add widgets and/or explore the ASM interface further.

This concludes the BOT Protection section of this lab guide!