Getting Started
I decided to branch this off into its own day, since it's kind of removed from what I was doing before.
I had decided to build a FreeRADIUS config from scratch so I could remove any unnecessary options that were included in the (rather large) default config file, and set my own baseline configuration to move forward.
Not too far into the configuration, I encountered log{} directive, which gave me a handful of logging options... including logging to syslog.
It occurred to me at this point that it might be worth actually setting up a syslog server, considering the breadth of all the things I'm trying to do, so I can have a single place to review log files.
So I did some research and settled on the free version of Splunk (partly because that seems like a helpful skillset in the outside world.)
Installation
First minor complication: virtually everything I've installed so far on CentOS has been available through the default yum repositories. Splunk is not - it comes as an RPM (or a .tgz, or a .deb package.) yum does use RPMs, but normally they're downloaded from repositories.
Anyways. I downloaded the RPM from Splunk to /root, and from there, it wasn't much different from a normal yum install - except for specifying a filename instead of a package name.
splunk installs slightly different from most packages, by putting everything under /opt/splunk. Not a huge deal, except that /opt/splunk/bin is not in my search path, so I just symlinked /usr/bin/splunk to /opt/splunk/bin/splunk and started up splunk per the install instructions.
Troubleshooting
At this point, I hit a minor snag: I couldn't access the Splunk web interface. I could ping the machine, and use SSH, I just couldn't use the web interface.
After some digging, I discovered something I was not aware of: specifically, that CentOS ships with its built-in firewall enabled, and the firewall's default settings only permit incoming SSH and DHCPv6 traffic.
Fortunately, an easy fix, once I found the right ports:
firewall-cmd --new-service=splunk --permanent
firewall-cmd --service=splunk --add-port=8000/tcp --permanent
firewall-cmd --service=splunk --add-port=8089/tcp --permanent
firewall-cmd --service=splunk --add-port=9997/tcp --permanent
firewall-cmd --reload
firewall-cmd --add-service=splunk
Perfect.
Configuration
The big "Add Data" button on the homepage seemed enticing, so I started down that path.
I didn't have any preexisting log files I wanted to import, and I didn't want to set up a forwarder - I just wanted Splunk to listen for incoming syslog data, so I went with Monitor.
The next screen continued on with the trend of being helpful:
At this point, I realized that I was going to have to modify my firewall rules again to accept incoming data:
firewall-cmd --service=splunk --add-port=514/tcp --permanent
firewall-cmd --reload
Naturally, when I did this, I couldn't access Splunk any more. Easy fix: I'd forgotten to add the --permanent option when I added the Splunk service to the current zone.
So, now, Splunk said it was ready to start receiving data. Linux has a utility called "logger" for making syslog entries, so I did exactly that:
# logger -T -n 127.0.0.1 -P 514 "test message"
Lo and behold - searching for "*" in Splunk found my message!
Next Up
Now that I have a syslog server working, it's back to configuring FreeRADIUS. Eventually I'll start migrating most of my CentOS services over to Splunk for logging.
