Splunk SDK for Python - Connecting to Splunk

I am going to try and cover my experience with the Splunk documentation here.

Connecting to Splunk via .py

If you have just installed the Splunk SDK for Python and you want to know how to connect to Splunk in the Python interpreter or through a python script, this is for you.

I used the saved_searches.py example as a starting place to see how python connects to Splunk.

Step 1) Use your preferred editor to modify ~/.splunkrc to contain your corresponding username, password, port and other configuration settings.

Note: "Storing login credentials in the .splunkrc file is only for convenience during development—this file isn't part of the Splunk platform and shouldn't be used for storing user credentials for production. And, if you're at all concerned about the security of your credentials, just enter them at the command line and don't bother using the .splunkrc file."

Step 2) Once you have that, test to make sure your authentication is correct by using the Splunk-Python interpreter located in the /examples/ folder:

 $ python spcmd.py

Welcome to Splunk SDK's Python interactive shell

admin connected to localhost:8089

>

If you can successfully connect there without passing any credentials at the command line than your .splunkrc file is set up correctly and you can continue. 

Step 3) Use your preferred editor to create a .py file.

Step 4) I used the saved_searches.py example as a template:

import sys, os

sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))

from splunklib.client import connect

try:

    from utils import parse

except ImportError:

    raise Exception("Add the SDK repository to your PYTHONPATH to run the examples "

                    "(e.g., export PYTHONPATH=~/splunk-sdk-python.")

def main():

    opts = parse(sys.argv[1:], {}, ".splunkrc")

    service = connect(**opts.kwargs)

    for app in service.apps:

        print app.name

if __name__ == "__main__":

    main()

Step 5) chmod 755 that .py file and execute it with: python filename.py

It should display all the installed apps on your Splunk instance. 

If it gives an error but you've already added PYTHONPATH to your splunk-sdk-python, try moving the .py file to a different folder. I store mine in ~/splunk-splunk-sdk-python/scripts/.

This is a brief example on how I connected to Splunk with Python. If you have a better way to do it, please share in the comments!

Splunking SSH failures

 If your SSH logs look like this:

Feb 14 22:35:49 foo-chassis66-blade66 sshd[31513]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.66.66  user=bar


And you want them to look like this (pardon the formatting):

Use this Splunk query:

index="Linux" process="sshd" | stats count values(rhost) values(host) by user | lookup dnslookup clientip as values(rhost) OUTPUT clienthost as Source_Hostname | table user,values(rhost),Source_Hostname,values(host)


Let's break this down:

index="Linux" - Searches my index known was Linux.

process="sshd" - Looks at the field process and only grabs events from the sshd process.

stats count values(rhost) values(host) by user - By using the values command, I can see all of the (plural) hosts the user attempted to connect from AND all of the hosts they failed to connect to.

lookup dnslookup clientip as values(rhost) OUTPUT clienthost as Source_Hostname - Using the lookup dnslookup function, I can have Splunk automatically perform a DNS lookup on the source host. This is good for reporting/alerting; it is easier to read "John's Mac" instead of "192.168.66.67".

table user,values(rhost),Source_Hostname,values(host) - The table spits out the user, all of the source host(s) they attempted to connect from, the DNS name of those host(s), and the hosts the user attempted to connect to.


Got a better way of doing it? Please share in the comments!