How to use Regex expressions when working with AWS WAF CloudFormation template

By Toul DeGuia-Cranmer

DevOps Engineer
HP Inc. in Houston

AWS announced support for using Regex Expressions for their WAF CloudFormation Templates. However, a quick prayer to the lord of IT – Google –  reveals that no one has figured it out from a solely CloudFormation solution.

This post tells you what can (and cannot) be done through editing the CloudFormation WAF template, which I discussed earlier in: 

How to Add OWASP 10 to a Load Balancer for a Kubernetes Cluster and EC2 Instances

For this post, I went down the AWS CloudFormation documentation rabbit hole and sort of figured it out. 

Reddit users (TL;DR): It is not possible to use CloudFormation to fully provision a Regex Rule for a Web ACL, but you can provison one part of it – the RegexPatternSet.

---

Quick Links

• Part I
  • a. Adding a RegexPatternSet via CloudFormation Template
  • b. Provisioning the RegexPatternSet
  • c. Trying the Provided Example
• PART II
  • Adding RegexPatternSet to RegexMatchSet through AWS CLI
• Part III
  • Creating a New Regex Rule for Web ACL
• Part IV
  • Adding Rule to Web ACL or WAF
• Common Gotchas

PART I

1a. Adding a RegexPatternSet via CloudFormation Template

First, open the OWASP template and add the RegexPatternSet, which conforms to the Java regular expression syntax. Note: I have omitted parts of the template for brevity. Also, I have deleted parts of the template that aren't necessary if you are going to be using the WAF on an Application Load Balancer, mainly the isGlobal pieces, which would be relevant if the WAF were going to be used for a CloudFront distribution. If you plan to use it for that, then do not delete isGlobal code.

## AWS OWASP 10 Template
## New RegexPatternSet
  wafRegexPatternSet:
    Type: AWS::WAFRegional::RegexPatternSet
    Condition: isRegional
    Properties:
      Name: !Join ['-', [!Ref stackPrefix, 'regex-rfi-lfi-traversal']]
      RegexPatternStrings:
        -
          "[B[a@]dB[o0]t"
        -
          "D[a@]ng[e3]rStr[i1]ng"

1b. Provisioning the RegexPatternSet

Storing the ID in a variable from the command line.

>$ REG_PAT_ID=

1c. Trying the Provided Example

According to the docs, it is not possible to add the newly created *RegexPatternSet* to a RegexMatchSet through CloudFormation, only through _"AWS WAF console, API, or command line interface (CLI)". That page may lead you to believe that it is possible through the code snippets shown, but it is not possible. Using the code snippets verbatim will result in the dreaded dead-end error of:

CREATE_FAILED The specified name is not permitted. (Service: AWSWAFRegional; Status Code: 400; Error Code: WAFDisallowedNameException; Request ID: ...

No matter what variation of the name is used or if an entirely new WAF is created. Therefore, the way to accomplish this is to use the AWS CLI tool.

PART II

Adding RegexPatternSet to RegexMatchSet through AWS CLI

First, create the RegexMatchSet.

>$ CHANGE_TOKEN=aws waf get-change-token
>$ aws waf create-regex-match-set --name badRobotMatchSet --change-token $CHANGE_TOKEN

Which, will output the ID of the badRobotMatchSet. Make sure to store it.

>$ REG_MATCH_SET_ID=

Second, update the RegexMatchSet by inserting the RegexPatternSet provisioned in CloudFormation.

>$ CHANGE_TOKEN=aws waf get-change-token
>$ aws waf update-regex-match-set --regex-match-set-id $REGEX_MATCH_SET_ID --updates Action="INSERT",RegexMatchTuple={FieldToMatch={Type=URI,DataId="myRegexMatchSet"},TextTransformation="LOWERCASE,RegexPatternSetId=$REG_PAT_ID}  --change-token $CHANGE_TOKEN

Voilà, now there is a RegexMatchSet that uses the CloudFormation-provisioned RegexPatternSet to filter URI parts of the httpRequest object to see if there are any "bad robots".

PART III

Creating a New Regex Rule for Web ACL

First provision the rule.

>$ CHANGE_TOKEN=aws waf get-change-token
>$ aws waf create-rule --name badRobotRegex --metric-name detect-bad-robot-regex --change-token
>$ BAD_ROBOT_RULE_ID=$

Then update the rule to be a RegexMatch.

>$ CHANGE_TOKEN=aws waf get-change-token
>$ aws waf update-rule --rule-id $BAD_ROBOT_RULE_ID --change-token $CHANGE_TOKEN --updates Action="INSERT",Predicate={Negated=false,Type="RegexMatch",DataId="badRobotRegex"}

PART IV

Finally, add the newly created Regex Rule to the pre-existing Web ACL.

>$ YOUR_WEB_ACL_ID=
>$ CHANGE_TOKEN=aws waf get-change-token
>$ aws waf update-web-acl --web-acli-id $YOUR_WEB_ACL_ID --change-token $CHANGE_TOKEN --updates Action="INSERT",ActivatedRule={Priority=10,RuleId=$BAD_ROBOT_RULE_ID,Action={Type="COUNT"}, Type="REGULAR"}
&_copy_id>

The Gotchas

There are limits to Regex for each account so be sure to keep that in mind.

Rules	limit
In regex match conditions the number of characters in the pattern that you want AWS WAF to search for	70
In regex match conditions the number of patterns per pattern set	10
In regex match conditions the number of pattern sets per regex condition	1
The number of pattern sets per account	5

Wrapping it up

Now, go to CloudFormation and either update an existing stack or provision a new one, with the newly added `RegexPatternSet` yaml added above.

After, doing so go to WAF & Shield > dropdown > select region > select Web ACL > String and regex matching > View regex pattern sets

And voilà, now you have a `RegexPatternSet` that is provisioned with a CloudFormation template for your AWS WAF as a condition. Copy the ID and set it as a variable, as it will be needed in Part 2. Adding RegexPatternSet to RegexMatchSet through AWS CLI.

---

More about me...

I interned at HP in 2018 while finishing a double-degree in Geophysics and Computer Science at the University of Houston. During undergrad, I studied abroad in Valencia, Spain and Stavenger, Norway. The highlight of my time in Europe was a month-long holiday through Barcelona, Paris, London, Stockholm, Cologne, and Amsterdam.

After graduation, I transitioned into my dream career as a DevOps Engineer at HP, Inc. where I am currently growing into a security focused DevOps Engineer. (Some would say that makes me a DevSecOps Engineer.) One of my first security improvement projects is auditing Docker and Kubernetes--two tools at the crux of our Continous Integration/Continous Deployment pipeline. By improving the security on those two tools in the toolchain I hope to further support Dion Weisler's claim of the world's most secure PC.

In my spare time, I like to apply my degree in Geophysics to suspenseful personal challenges.