I fought FreeRADIUS and FreeRADIUS won

Back to work

So... last I had written, several months ago, I had managed to add my CentOS server to my domain and was going to work on getting FreeRADIUS running there, so I could have my Cisco virtual devices authenticate against it.

Turns out, this was WAY more complicated then I thought it would be.

I'll spare the gory details, but long story short, I don't like the way FreeRADIUS tries to do... a lot of things.

Luckily, I complained about this on IRC, and someone else asked why I wasn't just using TACACS+. The reason I hadn't is because I don't have access to Cisco's TACACS+ server software - and I assumed there wasn't a free implementation of it out there.

As it turns out, when you assume, you may waste several months on a wild and unnecessary goose chase.

TACACS.net

There is indeed a free TACACS+ implementation, available from tacacs.net.

It's also ridiculously easy to set up. In addition to the normal configuration options (IP to bind to, mainly), all I had to do was specify:

authentication.xml

<UserGroup>

<Name>Router Access</Name>

<AuthenticationType>Windows_Domain</AuthenticationType>   

<LDAPServer>outpost-hub.outpost.amateria:636</LDAPServer>

<LDAPUseSSL>1</LDAPUseSSL>

<LDAPUserDirectorySubtree>OU=Accounts,DC=outpost,DC=amateria</LDAPUserDirectorySubtree> 

<LDAPGroupName>Router Access</LDAPGroupName>

</UserGroup>

clients.xml

<ClientGroup Name="INTERNAL">

<Secret ClearText="hidden" DES=""></Secret>

<Clients>

<Client>172.*</Client>

<Client>10.*</Client>

</Clients>

</ClientGroup>

authorization.xml

<Authorization>

<UserGroups>

<UserGroup>Router Access</UserGroup>

</UserGroups>

<ClientGroups>

<ClientGroup>INTERNAL</ClientGroup>

</ClientGroups>

<Shell>

<Permit>.*</Permit>

</Shell>

</Authorization>

And... that's it. In a nutshell, it takes requests from TACACS+ clients in the 172.* and 10.* ranges, looks up user accounts in the "OU=Accounts" DN, and confirms they belong to the "Router Access" LDAP group, and grants them permission to all commands on the router. (I could've made it more granular, but didn't see a need at this point.)

As a bonus, since it runs on a Windows host inside the domain, it can use the host's Kerberos credentials to access ActiveDirectory!

Bringing it online... jk

At this point, I went to configure one of my virtual Cisco routers to use TACACS for login, and... it didn't work.

After some digging, I found that for some reason, my CentOS and Windows Server machines couldn't route to the GNS3 LAN any more.

At this point, though, I decided that the original mesh design I had come up with, was probably overkill for what I was learning at the time. And also that it would probably need redesigning in the future when I learned new stuff.

So I killed the whole setup and went to a single router for now.

Also, I configured the input VTY lines for SSH.

At this point, I found something unusual - which is that trying to actually login to the router in question would hang for about 30-60 seconds, and then fail.

Coincidentally, at the same time, the CPU usage on the GNS3 VM would shoot up to about 100% and stay there. Nothing I seemed to do (switching from SSH to Telnet authentication) seemed to make a difference...

So for now, I'm just assuming that TACACS+/LDAP authentication would be working in a real environment, and moving on - because I've wasted more time on this than I'd like to admit.

I'm not really what I'll be moving on to next, but, we'll see.

Update: Back to work

Status update...

I've been busy for the last month-and-change with moving and finishing school.

Well, I've finished school, so now I'm getting back to work.

I'm still pressing with FreeRadius/LDAP authentication, so that'll be the next update whenever I get it done.

pyhakon.net goes up

I haven't posted on here in awhile.

I moved about a month back, and I've been busy with that and school and such, so I haven't had time to work on this project much.

But I just set up pyhakon.net to be the counterpart to this blog, where I plan to eventually put up information (in a much shorter and less narrative format) on setting up these kinds of adventures for yourself.

Day 4: Going Splunking

Getting Started

I decided to branch this off into its own day, since it's kind of removed from what I was doing before.

I had decided to build a FreeRADIUS config from scratch so I could remove any unnecessary options that were included in the (rather large) default config file, and set my own baseline configuration to move forward.

Not too far into the configuration, I encountered log{} directive, which gave me a handful of logging options... including logging to syslog.

It occurred to me at this point that it might be worth actually setting up a syslog server, considering the breadth of all the things I'm trying to do, so I can have a single place to review log files.

So I did some research and settled on the free version of Splunk (partly because that seems like a helpful skillset in the outside world.)

Installation

First minor complication: virtually everything I've installed so far on CentOS has been available through the default yum repositories. Splunk is not - it comes as an RPM (or a .tgz, or a .deb package.) yum does use RPMs, but normally they're downloaded from repositories.

Anyways. I downloaded the RPM from Splunk to /root, and from there, it wasn't much different from a normal yum install - except for specifying a filename instead of a package name.

splunk installs slightly different from most packages, by putting everything under /opt/splunk. Not a huge deal, except that /opt/splunk/bin is not in my search path, so I just symlinked /usr/bin/splunk to /opt/splunk/bin/splunk and started up splunk per the install instructions.

Troubleshooting

At this point, I hit a minor snag: I couldn't access the Splunk web interface. I could ping the machine, and use SSH, I just couldn't use the web interface.

After some digging, I discovered something I was not aware of: specifically, that CentOS ships with its built-in firewall enabled, and the firewall's default settings only permit incoming SSH and DHCPv6 traffic.

Fortunately, an easy fix, once I found the right ports:

firewall-cmd --new-service=splunk --permanent

firewall-cmd --service=splunk --add-port=8000/tcp --permanent

firewall-cmd --service=splunk --add-port=8089/tcp --permanent

firewall-cmd --service=splunk --add-port=9997/tcp --permanent

firewall-cmd --reload

firewall-cmd --add-service=splunk

Perfect.

Configuration

The big "Add Data" button on the homepage seemed enticing, so I started down that path.

I didn't have any preexisting log files I wanted to import, and I didn't want to set up a forwarder - I just wanted Splunk to listen for incoming syslog data, so I went with Monitor.

The next screen continued on with the trend of being helpful:

At this point, I realized that I was going to have to modify my firewall rules again to accept incoming data:

firewall-cmd --service=splunk --add-port=514/tcp --permanent

firewall-cmd --reload

Naturally, when I did this, I couldn't access Splunk any more. Easy fix: I'd forgotten to add the --permanent option when I added the Splunk service to the current zone.

So, now, Splunk said it was ready to start receiving data. Linux has a utility called "logger" for making syslog entries, so I did exactly that:

# logger -T -n 127.0.0.1 -P 514 "test message"

Lo and behold - searching for "*" in Splunk found my message!

Next Up

Now that I have a syslog server working, it's back to configuring FreeRADIUS. Eventually I'll start migrating most of my CentOS services over to Splunk for logging.

Day 3.5: ActiveDirectory and CentOS

Getting started... again

So World of Warcraft: Battle for Azeroth dropped last week, and that's been taking up a lot of time that I might otherwise have been using to work on this project... but I hit level 120 on my main, so now I'm spending less time on that.

Anyways. Last time, I had decided I needed to do some reading and figure out the underlying mechanics of tying Cisco devices to ActiveDirectory, which as best I could tell, was going to require a long chain of services including FreeRADIUS, Samba, Kerberos, and ActiveDirectory. Along the way, I decided now would be as good a time as any to transition my CentOS server from using local accounts to using ActiveDirectory user accounts.

I thought this was going to require a whole bunch of manual Samba and Kerberos setup, and thought I had it all figured out... and then when I went to research the actual configuration, I discovered it was actually going to be a lot simpler.

CentOS setup

Red Hat has done a pretty decent job of consolidating core system services into easily managed software packages. Originally, my research suggested I was going to have to configure a handful of different Samba processes running on top of Kerberos, and configure the system by hand... but as it turns out, CentOS has made this a lot easier with two particular services, realmd and sssd. realmd essentially handles the process of connecting the server to an ActiveDirectory domain, and sssd handles interfacing between clients and back-end authentication mechanisms (like ActiveDirectory.)

realmd wasn't installed by default, but it was available through yum, so installing it was easy enough.

Following the documentation, I was able to discover my domain, and then tried to join it:

I could've sworn I saw those packages listed as dependencies when I installed realmd, but apparently not. But, again, easy enough to fix...

# yum install oddjob oddjob-mkhomedir sssd adcli

Oddly, during this process, yum determined that oddjob/oddjob-mkhomedir were already installed (I didn't imagine it!) but the other two weren't, so I opted to install them and then deal with the "missing" packages afterwards.

Impressively, the 28-package install took about 5 seconds.

And, as a pleasant surprise - after installing the two missing packages, it didn't complain at all when I went to join the domain again.

Equally pleasantly was going to test the new implementation, as described in the documentation, and finding that realmd had worked exactly as advertised and had taken care of the entire joining process (including configuring sssd for ActiveDirectory) for me.

(At this point, I also went into the ActiveDirectory Users/Computers tool and set my password not to expire.)

At this point, I wanted to make it even more fancy... and make sure that only designated users could use ActiveDirectory credentials to log into the CentOS server.

So I jumped over to Windows and created a new user group: 

And then used the appropriate realmd commands to block access from anyone except that user group:

First I tested it without adding myself to that user group:

Then added myself through Windows and tried again:

Perfection. And amazingly easy, too.

Next up

Next up is to configure FreeRADIUS to authenticate against ActiveDirectory. I was originally going to make that part of this post, but it's looking complicated and I need to read up before I continue. Until next time...

Break time: RADIUS, ActiveDirectory, and everything in between

Status update

The next thing I had planned was to link my virtual Cisco devices to my Windows domain, so I could do a couple things: control login access to the devices themselves, and set up 802.lx authentication for connected VMs.

There's a couple things I need to do for that to happen:

1. Set up a RADIUS server for the Cisco devices to talk to.
2. Set up an SMB server to act as a "go-between" between the RADIUS server, and the Windows domain controller.

At this point, though, I'm solidly in the realm of protocols and software that I have very little understanding of: FreeRADIUS to implement RADIUS, Samba/Kerberos to implement SMB, and configuring Samba to talk to the domain controller via LDAP 

So for now, I've made myself a Kindle reading list on Samba/Kerberos/LDAP so I can get an understanding of all the pieces I'm trying to put together.

So once I've gotten through that, I'll start attacking the actual setup.

More to come...

Day 3.4: DHCP and subnetting

Introduction

In the last entry, I set up a few things:

• Routing between all the virtual routers on the network.
• To do this, I enabled RIP on all the core (Core, Edge, and Access) routers for all the networks in the LAN.
• Configured the Core0 router as the default gateway, and configured RIP to propagate default route information across the network.
• Configured the Win2016 VM with a direct connection to the Internet, since while GNS3 routes traffic internally quickly enough, it's too slow for Internet connectivity. Win2016 still has access to the virtual LAN as well.

Onwards!

Next step is going to be setting up user access and VLANs (which, by necessity, involves configuring DHCP.)

But VLANs first.

Jumping over to my Access1 router, I set up my two user VLANs, which I designated 101 and 102:

Access1(vlan)#vlan 101

VLAN 101 added:

    Name: VLAN0101

Access1(vlan)#vlan 102

VLAN 102 added:

    Name: VLAN0102

Immediate problem: VLANs are apparently stored in Flash memory as opposed to NVRAM... and Access1 is not configured with any Flash memory.

Shut down switch. Go into configuration in GNS3. Add an 8MiB PCMCIA disk. Restart switch.

format flash:

Okay... now I can create my VLANs and update the database.

Next up: configure an IP address for each VLAN interface to use.

Access1(config)#interface Vlan101

Access1(config-if)#ip address 10.100.1.1 255.255.255.0

Access1(config-if)#no shut

Access1(config-if)#exit

Access1(config)#interface Vlan102

Access1(config-if)#ip address 10.100.2.1 255.255.255.0

Access1(config-if)#no shut

Access1(config-if)#exit

Access1(config)#

Since my VLANs aren't crossing multiple switches, I don't need to work about trunking to other switches. All I need to do is configure the appropriate Ethernet ports for the appropriate VLANs. GNS3 tells me in the Topology Summary which VMs are connected to which ports:

Access1#config t

Enter configuration commands, one per line.  End with CNTL/Z.

Access1(config)#int fa3/0

Access1(config-if)#switchport access vlan 101

Access1(config-if)#ex

Access1(config)#

*Mar  1 00:20:00.015: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan101, changed state to up

Access1(config)#int fa3/1

Access1(config-if)#switchport access vlan 102

Access1(config-if)#^Z

Access1#

And...

Access1#show vlan-switch

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

101  VLAN0101                         active    Fa3/0

102  VLAN0102                         active    Fa3/1

All right, VLANs are set. One more thing first, configure each VLAN interface to use the Windows 2016 DHCP server.

Access1(config)#int Vlan101

Access1(config-if)#ip helper-address 10.1.0.100

Access1(config-if)#ex

Access1(config)#int Vlan102

Access1(config-if)#ip helper-address 10.1.0.100

Access1(config-if)#^Z

Access1#

Now to jump over to Windows and set up the DHCP server.

There's a few different options that needed to be set up:

I created a Superscope specifically to cover user VLANs, and then added a Scope for each VLAN to assign IP addresses within that subnet.

I also set a couple different server-wide options to control what gets assigned to all DHCP clients:

The two Router addresses are for the two interfaces of Core0, the DNS server is the IP address of the Windows 2016 VM.

Time to boot up my Windows 8 VM and see if it gets an IP address successfully.

Probably should've taken care of this first. Oh well.

After the VM came up, I discovered I was getting an address assigned by VMware. As it turns out, you have to start the VMs within GNS3 in order for GNS3 to control them.

But when I tried to do that, I got this message:

After a little digging, I discovered that GNS3 "takes control" of pre-configured VMware virtual networks while it's running. I had only configured it to use a single VMnet, and that was already in use by Win2016. After giving it control of several more VMnets and assigning Windows8 to one of those VMnets, I tried again:

I'll take "error messages I don't understand" for $400, Alex.

Oddly, the VM itself started up with no problem, although it had no network connectivity. I tried one more option: the "Allow GNS3 to override non custom VMware adapter," which I finally discovered had a tooltip informing me that this would allow GNS3 to control the network adapters belonging to the VM.

Still nothing.

After doing some research, I found a forum post suggesting that Windows-based clouds (which I did indeed have) were not as reliable as clouds running inside the GNS3 VM. Following the post's suggestion, I added a network adapter to the GNS3 VM, bridged to my home LAN, and rebooted the GNS3 VM.

GNS3 itself apparently did not like that I was changing its VM while it was running, and complained when I tried to add a new cloud. I opted to just restart GNS3 and try again.

Success... except I was now getting that error message when trying to start up Windows 2016.

At this point I noticed that when I was trying to configure VMnets in GNS3, it was trying to start two services related to Netgroup Packet Filter, that appeared to start and then stop. I figured that all my network interface shifting might have broken something on my local machine, so I decided to reboot.

I started up Win2016, so far so good - then the rest of my virtual routers - then Windows 8. All well and good, except... no IP address.

Oops. Apparently I forgot to save my Access1 configuration before restarting.

After rebuilding the configuration... no DHCP.

Time to enable UDP debugging on Access1 and try a DHCP renew:

Access1#debug ip udp
UDP packet debugging is on
Access1#
*Mar  1 00:07:10.863: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=308
*Mar  1 00:07:10.867: UDP: sent src=10.100.2.1(67), dst=10.1.0.100(67), length=308

UDP packets are making it out, but not coming back. Checking on the next hop, InternalEdge2... no UDP packets are coming through, and yet, I can tracert to the DHCP server just fine.

Turning on IP error debugging on InternalEdge2 reveals:

InternalEdge2#debug ip error

IP packet errors debugging is on

InternalEdge2#

*Mar  1 00:31:01.927: IP: s=10.1.0.100 (FastEthernet2/0), d=10.100.2.1, len 337, dispose ip.noroute

There's the problem. I never added my user subnets to the RIP network listing on Access1, and so the rest of the network has no idea how to reach those subnets. Easy fix, at least:

Access1#config t

Enter configuration commands, one per line.  End with CNTL/Z.

Access1(config)#router rip

Access1(config-router)#network 10.100.0.0

Access1(config-router)#^Z

Access1#

Checking the route listing on InternalEdge2 now shows a route to the 10.100.0.0 networks. DHCP renewed, and...

Got it!

Next step

Not sure, really. I'll figure something out.