Integrating TechPulse Data with Splunk By Creating a Splunk App

Questions about this tutorial?

Reach out to us in our Forum: TechPulse API Forum

Get your refresh token

Use Postman or our Python Examples to get your refresh token. We will use the refresh token to request an access token from the API.

If you need help with this then reach out to us in our TechPulse API Forum

Keep it secret. Keep it safe.

For this example Splunk App we are going store our credentials in the script itself.

This is not a best practice in any way!

If you implement this in your own environment then you will need to look at storing your credentials using one of the following suggested ways:

  • environmental variables accessible to Splunk
  • using Splunk's built in methodology for storing secrets combined with setup.xml
  • using some other secrets storing platform

Setting Up Our Scripts

By default Splunk uses Python 2 to execute user created Python scripts.

Using Python 2 these days is not a great idea so we are going to create a wrapper script using bash to call our script with Python 3.

Find out where Splunk is installed. For example doing this on my local machine it is setup in /Applications/Splunk/

Throughout the rest of this tutorial I will use $SPLUNK_HOME in place of whatever folder your Splunk is installed in.

Setup the wrapper script

# this is where we store our scripts
cd $SPLUNK_HOME/bin/scripts/ 
touch techpulse.sh

Now Insert the following text into techpulse.sh using your favorite text editor

# techpulse.sh

# Be sure to replace this with your specific splunk path
SPLUNK_HOME=/Applications/Splunk

cd $SPLUNK_HOME/bin/scripts/
python3 $SPLUNK_HOME/bin/scripts/techpulse2.py

Setup creds.json

creds.json is what we will use to persist our refresh tokens for this simple example application.

Note: Once again this is not a production ready solution. In a production application you will need a more secure way to store, retrieve, and reuse credentials.

Use the following template for your creds.json

{
    "access_token": "",
    "scope": "Read",
    "token_type": "Bearer",
    "refresh_token": "",
    "expires_in": 3599
}

Be sure to replace refresh_token with your latest refresh token that you got above.

Setup the Python Script techpulse.py

Now we need to setup the script that will actually pull the data.

# techpulse.py
import requests
import json
import base64
import sys


# Insert your own info here as you defined when you created your APP
# Note that in a real app you would not want to hard-code these values
# Instead you would want to import them from the environment or even use
# a more secure solution like a keystore.
CLIENT_ID = "" 
CLIENT_SECRET = ""
REDIRECT_URI = ""
STATE = "yourstatestring"  # note that in a true production app you would use state to protect against cross site attacks
HOSTNAME = 'daas.api.hp.com'

def refresh_access_token():
    '''use refresh token to get a new access token'''


    with open('creds.json', 'r') as f:
        creds = json.load(f)
    refresh_token = creds['refresh_token']


    base64_encoded_clientid_clientsecret = base64.b64encode(str.encode(f'{CLIENT_ID}:{CLIENT_SECRET}'))  # concatenate with : and encode in base64
    base64_encoded_clientid_clientsecret = base64_encoded_clientid_clientsecret.decode('ascii')  # turn bytes object into ascii string

    base_url = f'https://{HOSTNAME}'
    url = f"{base_url}/oauth/v1/token"
    headers = {
        'Content-Type': "application/x-www-form-urlencoded",
        'Authorization': f'Basic {base64_encoded_clientid_clientsecret}'
        }

    data = {'grant_type': 'refresh_token',
            'redirect_uri': REDIRECT_URI,
            'refresh_token': refresh_token
            }

    r = requests.post(url, headers=headers, data=data)
    response = r.json()

    if response.get('access_token'):
        # don't store creds in plaintext in a real app obviously
        with open('creds.json', 'w') as f:
            json.dump(response, f, indent=4)

        return response.get('access_token')

    else:
        print('There was an error refreshing your access token')
        print(r.text)
        sys.exit(1)


def get_device_security_compliance(access_token):

    base_url = f'https://{HOSTNAME}'
    url = f"{base_url}/analytics/v1/reports/devicesec/twentyFourHrSummary/type/grid"
    headers = {
        'Content-Type': "application/json",
        'Authorization': f"Bearer {access_token}",
        }

    # get total results for this query so we know how much data we need
    response = requests.request("POST", url, headers=headers, params={'count': 1})
    total_results = int(response.json()['totalResults'])

    # Now time to get all the results from the API
    start_index = 1
    results = []
    current_results = 0

    # paginate through the results and get them all
    while current_results < total_results:

        querystring = {"startIndex": f"{start_index}",
                       "count": "1000"}
        response = requests.post( url, headers=headers, params=querystring)
        response = response.json()

        results += response['resources']
        current_results = len(results)

        start_index += 1

    print(json.dumps(results, indent=4))

    return 0



if __name__ == '__main__':
    access_token = refresh_access_token()
    get_device_security_compliance(access_token)

Summary

You should now have the following scripts in $SPLUNK_HOME/bin/scripts/

creds.json - used to persist our creds for this example
readme.txt - Auto gen by Splunk
techpulse.py - main python script that pulls from API
techpulse.sh - wrapper script

Create a new Splunk Data Input

In your Splunk Web UI Click on "Data inputs"

tutorial/Untitled.png

Then click on add new local script

tutorial/Untitled%201.png

From there click "New Local Script"

Now fill out the following to match what you see below. For this example we are doing every 20 seconds but you can do once a day, etc...

tutorial/Untitled%202.png

Now setup the following properties like we did here or modify them depending on your use case:

tutorial/Untitled%203.png

Click Review

Click Submit

Click Start Searching

Success!

You should now see data like the following being input into your Splunk UI

tutorial/Untitled%204.png