pyhakon.net goes up

I haven't posted on here in awhile.

I moved about a month back, and I've been busy with that and school and such, so I haven't had time to work on this project much.

But I just set up pyhakon.net to be the counterpart to this blog, where I plan to eventually put up information (in a much shorter and less narrative format) on setting up these kinds of adventures for yourself.

Day 4: Going Splunking

Getting Started

I decided to branch this off into its own day, since it's kind of removed from what I was doing before.

I had decided to build a FreeRADIUS config from scratch so I could remove any unnecessary options that were included in the (rather large) default config file, and set my own baseline configuration to move forward.

Not too far into the configuration, I encountered log{} directive, which gave me a handful of logging options... including logging to syslog.

It occurred to me at this point that it might be worth actually setting up a syslog server, considering the breadth of all the things I'm trying to do, so I can have a single place to review log files.

So I did some research and settled on the free version of Splunk (partly because that seems like a helpful skillset in the outside world.)

Installation

First minor complication: virtually everything I've installed so far on CentOS has been available through the default yum repositories. Splunk is not - it comes as an RPM (or a .tgz, or a .deb package.) yum does use RPMs, but normally they're downloaded from repositories.

Anyways. I downloaded the RPM from Splunk to /root, and from there, it wasn't much different from a normal yum install - except for specifying a filename instead of a package name.

splunk installs slightly different from most packages, by putting everything under /opt/splunk. Not a huge deal, except that /opt/splunk/bin is not in my search path, so I just symlinked /usr/bin/splunk to /opt/splunk/bin/splunk and started up splunk per the install instructions.

Troubleshooting

At this point, I hit a minor snag: I couldn't access the Splunk web interface. I could ping the machine, and use SSH, I just couldn't use the web interface.

After some digging, I discovered something I was not aware of: specifically, that CentOS ships with its built-in firewall enabled, and the firewall's default settings only permit incoming SSH and DHCPv6 traffic.

Fortunately, an easy fix, once I found the right ports:

firewall-cmd --new-service=splunk --permanent

firewall-cmd --service=splunk --add-port=8000/tcp --permanent

firewall-cmd --service=splunk --add-port=8089/tcp --permanent

firewall-cmd --service=splunk --add-port=9997/tcp --permanent

firewall-cmd --reload

firewall-cmd --add-service=splunk

Perfect.

Configuration

The big "Add Data" button on the homepage seemed enticing, so I started down that path.

I didn't have any preexisting log files I wanted to import, and I didn't want to set up a forwarder - I just wanted Splunk to listen for incoming syslog data, so I went with Monitor.

The next screen continued on with the trend of being helpful:

At this point, I realized that I was going to have to modify my firewall rules again to accept incoming data:

firewall-cmd --service=splunk --add-port=514/tcp --permanent

firewall-cmd --reload

Naturally, when I did this, I couldn't access Splunk any more. Easy fix: I'd forgotten to add the --permanent option when I added the Splunk service to the current zone.

So, now, Splunk said it was ready to start receiving data. Linux has a utility called "logger" for making syslog entries, so I did exactly that:

# logger -T -n 127.0.0.1 -P 514 "test message"

Lo and behold - searching for "*" in Splunk found my message!

Next Up

Now that I have a syslog server working, it's back to configuring FreeRADIUS. Eventually I'll start migrating most of my CentOS services over to Splunk for logging.

Day 3.5: ActiveDirectory and CentOS

Getting started... again

So World of Warcraft: Battle for Azeroth dropped last week, and that's been taking up a lot of time that I might otherwise have been using to work on this project... but I hit level 120 on my main, so now I'm spending less time on that.

Anyways. Last time, I had decided I needed to do some reading and figure out the underlying mechanics of tying Cisco devices to ActiveDirectory, which as best I could tell, was going to require a long chain of services including FreeRADIUS, Samba, Kerberos, and ActiveDirectory. Along the way, I decided now would be as good a time as any to transition my CentOS server from using local accounts to using ActiveDirectory user accounts.

I thought this was going to require a whole bunch of manual Samba and Kerberos setup, and thought I had it all figured out... and then when I went to research the actual configuration, I discovered it was actually going to be a lot simpler.

CentOS setup

Red Hat has done a pretty decent job of consolidating core system services into easily managed software packages. Originally, my research suggested I was going to have to configure a handful of different Samba processes running on top of Kerberos, and configure the system by hand... but as it turns out, CentOS has made this a lot easier with two particular services, realmd and sssd. realmd essentially handles the process of connecting the server to an ActiveDirectory domain, and sssd handles interfacing between clients and back-end authentication mechanisms (like ActiveDirectory.)

realmd wasn't installed by default, but it was available through yum, so installing it was easy enough.

Following the documentation, I was able to discover my domain, and then tried to join it:

I could've sworn I saw those packages listed as dependencies when I installed realmd, but apparently not. But, again, easy enough to fix...

# yum install oddjob oddjob-mkhomedir sssd adcli

Oddly, during this process, yum determined that oddjob/oddjob-mkhomedir were already installed (I didn't imagine it!) but the other two weren't, so I opted to install them and then deal with the "missing" packages afterwards.

Impressively, the 28-package install took about 5 seconds.

And, as a pleasant surprise - after installing the two missing packages, it didn't complain at all when I went to join the domain again.

Equally pleasantly was going to test the new implementation, as described in the documentation, and finding that realmd had worked exactly as advertised and had taken care of the entire joining process (including configuring sssd for ActiveDirectory) for me.

(At this point, I also went into the ActiveDirectory Users/Computers tool and set my password not to expire.)

At this point, I wanted to make it even more fancy... and make sure that only designated users could use ActiveDirectory credentials to log into the CentOS server.

So I jumped over to Windows and created a new user group: 

And then used the appropriate realmd commands to block access from anyone except that user group:

First I tested it without adding myself to that user group:

Then added myself through Windows and tried again:

Perfection. And amazingly easy, too.

Next up

Next up is to configure FreeRADIUS to authenticate against ActiveDirectory. I was originally going to make that part of this post, but it's looking complicated and I need to read up before I continue. Until next time...

Break time: RADIUS, ActiveDirectory, and everything in between

Status update

The next thing I had planned was to link my virtual Cisco devices to my Windows domain, so I could do a couple things: control login access to the devices themselves, and set up 802.lx authentication for connected VMs.

There's a couple things I need to do for that to happen:

1. Set up a RADIUS server for the Cisco devices to talk to.
2. Set up an SMB server to act as a "go-between" between the RADIUS server, and the Windows domain controller.

At this point, though, I'm solidly in the realm of protocols and software that I have very little understanding of: FreeRADIUS to implement RADIUS, Samba/Kerberos to implement SMB, and configuring Samba to talk to the domain controller via LDAP 

So for now, I've made myself a Kindle reading list on Samba/Kerberos/LDAP so I can get an understanding of all the pieces I'm trying to put together.

So once I've gotten through that, I'll start attacking the actual setup.

More to come...

Day 3.4: DHCP and subnetting

Introduction

In the last entry, I set up a few things:

• Routing between all the virtual routers on the network.
• To do this, I enabled RIP on all the core (Core, Edge, and Access) routers for all the networks in the LAN.
• Configured the Core0 router as the default gateway, and configured RIP to propagate default route information across the network.
• Configured the Win2016 VM with a direct connection to the Internet, since while GNS3 routes traffic internally quickly enough, it's too slow for Internet connectivity. Win2016 still has access to the virtual LAN as well.

Onwards!

Next step is going to be setting up user access and VLANs (which, by necessity, involves configuring DHCP.)

But VLANs first.

Jumping over to my Access1 router, I set up my two user VLANs, which I designated 101 and 102:

Access1(vlan)#vlan 101

VLAN 101 added:

    Name: VLAN0101

Access1(vlan)#vlan 102

VLAN 102 added:

    Name: VLAN0102

Immediate problem: VLANs are apparently stored in Flash memory as opposed to NVRAM... and Access1 is not configured with any Flash memory.

Shut down switch. Go into configuration in GNS3. Add an 8MiB PCMCIA disk. Restart switch.

format flash:

Okay... now I can create my VLANs and update the database.

Next up: configure an IP address for each VLAN interface to use.

Access1(config)#interface Vlan101

Access1(config-if)#ip address 10.100.1.1 255.255.255.0

Access1(config-if)#no shut

Access1(config-if)#exit

Access1(config)#interface Vlan102

Access1(config-if)#ip address 10.100.2.1 255.255.255.0

Access1(config-if)#no shut

Access1(config-if)#exit

Access1(config)#

Since my VLANs aren't crossing multiple switches, I don't need to work about trunking to other switches. All I need to do is configure the appropriate Ethernet ports for the appropriate VLANs. GNS3 tells me in the Topology Summary which VMs are connected to which ports:

Access1#config t

Enter configuration commands, one per line.  End with CNTL/Z.

Access1(config)#int fa3/0

Access1(config-if)#switchport access vlan 101

Access1(config-if)#ex

Access1(config)#

*Mar  1 00:20:00.015: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan101, changed state to up

Access1(config)#int fa3/1

Access1(config-if)#switchport access vlan 102

Access1(config-if)#^Z

Access1#

And...

Access1#show vlan-switch

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

101  VLAN0101                         active    Fa3/0

102  VLAN0102                         active    Fa3/1

All right, VLANs are set. One more thing first, configure each VLAN interface to use the Windows 2016 DHCP server.

Access1(config)#int Vlan101

Access1(config-if)#ip helper-address 10.1.0.100

Access1(config-if)#ex

Access1(config)#int Vlan102

Access1(config-if)#ip helper-address 10.1.0.100

Access1(config-if)#^Z

Access1#

Now to jump over to Windows and set up the DHCP server.

There's a few different options that needed to be set up:

I created a Superscope specifically to cover user VLANs, and then added a Scope for each VLAN to assign IP addresses within that subnet.

I also set a couple different server-wide options to control what gets assigned to all DHCP clients:

The two Router addresses are for the two interfaces of Core0, the DNS server is the IP address of the Windows 2016 VM.

Time to boot up my Windows 8 VM and see if it gets an IP address successfully.

Probably should've taken care of this first. Oh well.

After the VM came up, I discovered I was getting an address assigned by VMware. As it turns out, you have to start the VMs within GNS3 in order for GNS3 to control them.

But when I tried to do that, I got this message:

After a little digging, I discovered that GNS3 "takes control" of pre-configured VMware virtual networks while it's running. I had only configured it to use a single VMnet, and that was already in use by Win2016. After giving it control of several more VMnets and assigning Windows8 to one of those VMnets, I tried again:

I'll take "error messages I don't understand" for $400, Alex.

Oddly, the VM itself started up with no problem, although it had no network connectivity. I tried one more option: the "Allow GNS3 to override non custom VMware adapter," which I finally discovered had a tooltip informing me that this would allow GNS3 to control the network adapters belonging to the VM.

Still nothing.

After doing some research, I found a forum post suggesting that Windows-based clouds (which I did indeed have) were not as reliable as clouds running inside the GNS3 VM. Following the post's suggestion, I added a network adapter to the GNS3 VM, bridged to my home LAN, and rebooted the GNS3 VM.

GNS3 itself apparently did not like that I was changing its VM while it was running, and complained when I tried to add a new cloud. I opted to just restart GNS3 and try again.

Success... except I was now getting that error message when trying to start up Windows 2016.

At this point I noticed that when I was trying to configure VMnets in GNS3, it was trying to start two services related to Netgroup Packet Filter, that appeared to start and then stop. I figured that all my network interface shifting might have broken something on my local machine, so I decided to reboot.

I started up Win2016, so far so good - then the rest of my virtual routers - then Windows 8. All well and good, except... no IP address.

Oops. Apparently I forgot to save my Access1 configuration before restarting.

After rebuilding the configuration... no DHCP.

Time to enable UDP debugging on Access1 and try a DHCP renew:

Access1#debug ip udp
UDP packet debugging is on
Access1#
*Mar  1 00:07:10.863: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=308
*Mar  1 00:07:10.867: UDP: sent src=10.100.2.1(67), dst=10.1.0.100(67), length=308

UDP packets are making it out, but not coming back. Checking on the next hop, InternalEdge2... no UDP packets are coming through, and yet, I can tracert to the DHCP server just fine.

Turning on IP error debugging on InternalEdge2 reveals:

InternalEdge2#debug ip error

IP packet errors debugging is on

InternalEdge2#

*Mar  1 00:31:01.927: IP: s=10.1.0.100 (FastEthernet2/0), d=10.100.2.1, len 337, dispose ip.noroute

There's the problem. I never added my user subnets to the RIP network listing on Access1, and so the rest of the network has no idea how to reach those subnets. Easy fix, at least:

Access1#config t

Enter configuration commands, one per line.  End with CNTL/Z.

Access1(config)#router rip

Access1(config-router)#network 10.100.0.0

Access1(config-router)#^Z

Access1#

Checking the route listing on InternalEdge2 now shows a route to the 10.100.0.0 networks. DHCP renewed, and...

Got it!

Next step

Not sure, really. I'll figure something out.

Day 3.3: Setting up routing

Introduction

Yesterday I built a network of virtual routers in GNS3 and connected my VMs to them. Today I'm going to work on configuring them to actually route traffic around the virtual network.

Getting started

As a reminder, here's what the network looks like right now:

To start out, I figured I should configure Core0 to connect to the Internet. Essentially, I started with the instructions here to set my "external" interface on Core0 (Fa0/0) for DHCP:

Core0#

*Mar  1 00:13:24.639: %DHCP-6-ADDRESS_ASSIGN: Interface FastEthernet0/0 assigned DHCP address 172.20.179.154, mask 255.255.0.0, hostname Core0

Core0#

So far so good - Fa0/0 now has an IP address on my home LAN. Equally importantly, it now has a default gateway assigned for routing traffic out to the Internet.

Core0#ping 75.75.75.75

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 75.75.75.75, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

Okay, my bad. The default route forwards out to Fa0/0, but there's no default gateway set.

Core0(config)#ip route 0.0.0.0 0.0.0.0 172.20.0.1 permanent

Core0(config)#^Z

Core0#

*Mar  1 00:24:31.687: %SYS-5-CONFIG_I: Configured from console by console

Core0#sh ip route

Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP

       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

       E1 - OSPF external type 1, E2 - OSPF external type 2

       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

       ia - IS-IS inter area, * - candidate default, U - per-user static route

       o - ODR, P - periodic downloaded static route

Gateway of last resort is 172.20.0.1 to network 0.0.0.0

C    172.20.0.0/16 is directly connected, FastEthernet0/0

     172.25.0.0/24 is subnetted, 2 subnets

C       172.25.1.0 is directly connected, FastEthernet1/0

C       172.25.2.0 is directly connected, FastEthernet2/0

S*   0.0.0.0/0 [1/0] via 172.20.0.1

               is directly connected, FastEthernet0/0

Core0#ping 75.75.75.75

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 75.75.75.75, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 60/78/96 ms

Core0#

Much better.

Next step is configuring the rest of the routers to learn routes. At this point, for simplicity, I'm going to use RIP.

Fortunately, GNS3 lets me edit each router's startup-config in a text editor so I don't have to manually enter all the config commands on each router to add:

router rip
version 2
network 172.25.0.0
network 172.26.0.0
network 172.27.0.0
network 172.28.0.0
network 172.29.0.0
no auto-summary

Next up, configure Core0 to propagate its static route via RIP:

Core0(config-router)#default-information originate

A quick check of Core2 confirms that the default route has propagated:

Core2#sh ip rip database

0.0.0.0/0    auto-summary

0.0.0.0/0

    [1] via 172.25.2.1, 00:00:01, FastEthernet0/0

Unfortunately, I can't ping 75.75.75.75 from Core2.

Tracerouting shows that packets are making it to Core1 and no further.

Core2#traceroute 75.75.75.75

Type escape sequence to abort.

Tracing the route to 75.75.75.75

  1 172.25.2.1 64 msec 64 msec 60 msec

  2  *

Oops. Apparently GNS3 Clouds don't work the way I had thought - I need to enable NAT so that Core0 can translate packets between my virtual LAN and my home LAN.

After some quick changes (irrelevant lines omitted):

interface FastEthernet0/0

 ip nat outside

interface FastEthernet1/0

 ip nat inside

interface FastEthernet2/0

 ip nat inside

ip nat inside source list 1 interface FastEthernet0/0 overload

!

access-list 1 permit 172.25.0.0 0.0.255.255

access-list 1 permit 172.26.0.0 0.0.255.255

access-list 1 permit 172.27.0.0 0.0.255.255

access-list 1 permit 172.28.0.0 0.0.255.255

access-list 1 permit 172.29.0.0 0.0.255.255

I can now ping the outside world from Core2.

Next up: configure default route propagation between all core routers with default-information originate.

Even after doing this, route information is not making it to Access1.

After some digging, I discovered that neither Core1 or Core2 was sending RIP updates to Access1 because I'd somehow forgotten to activate that interface on both Core1 and Core2.

After fixing this, I tried to ping 75.75.75.75 from Access1 - it worked, but only with a 40-60% success rate. Same with most other Internet sites I tried.

Problematically, this doesn't appear to happen if I try it from my physical machine, and I don't have this problem with pings anywhere in my home LAN. It looks like the latency within GNS3 - while not a problem as long as I'm inside my own network - is too slow for traffic to the Internet.

This isn't really a problem, except that my Windows 2016 server needs to be able to access the Windows Update service, and it can't do that if packets are constantly timing out.

Fortunately, this should be a fairly easy fix - I can add a second network adapter to that VM in VMware, connect it directly to my home LAN (bypassing GNS3) and set that as the default route for all traffic not headed for my virtual LAN.

So at this point, my W2016 VM has two network connections: SIMULCRON (connected to my virtual LAN) and EXTERNAL (connected directly to my home LAN.)

(It was also at this point that I switched ExtEdge2 back to a two-port router, and added a generic switch connecting the Windows 2016 VM to it, for simplicity configuring the router.)

Here's what my routing table ended up looking like:

Oddly, at this point, I can access the Internet and ping ExtEdge2, but I can't reach the rest of the virtual LAN.

(Sidenote: I discovered during this process that GNS3 doesn't handle editing of startup-config via the text editor very well if you do it while the VND is running.)

Not too soon after, I realized I'd never configured ExtEdge2 to advertise its path to Win2016 (which is on the 10.1.0.0/16 network, rather than the rest of the virtual network, which is a 172.* network.) After adding 10.1.0.0 to the RIP database on ExtEdge2, everything worked perfectly - and I confirmed I can ping from Core0 to Win2016, and back!

Next up...

Next step will be setting up user VLANs, like I planned in the original network diagram.

As an aside, while network performance is fine, Win2016 is struggling to paint the screen when I'm doing command-line stuff. I may also start using VPCS simulators to test network connectivity.

Day 3.2 - Designing/building a LAN

Introduction

In the last couple posts I went through setting up GNS3 to act a network simulator to route traffic between my virtual machines.

(As an aside, I had to update my Windows 2016 VM last night and it took... a really long time. At some point, the number of VMs I'm running may become an issue - especially with the number of routers I'm putting into the GNS3 VM. But I'll cross that bridge if I get there.)

LAN Design

I'm not a professional, so I didn't make this TOO terribly fancy. (Although it ended up being fancier than I originally anticipated.)

I had a few objectives:

• Multiple VLANs to practice trunking/switching/etc.
• A DMZ outside the main network
• A "core" network with multiple routes, to practice with routing protocols

With that in mind, here's what I came up with.

I made it in a program called CADE, which is a little finicky. I didn't want to start putting things into GNS3 - which makes them "real" - until I had a layout in mind.

As far as end-user devices go, GNS3 has a feature called VPCS - or, Virtual PC Simulator. This allows you to create miniature console-based VMs that do the bare minimum to respond to network traffic - so you can test connectivity from an end-user perspective, without going to the trouble of building a full-fledged OS-based VM.

With that in mind, for my virtual network, I decided I'm probably going to only run a single actual VM for testing Windows domain-based stuff, and use VPCS for the rest.

Starting the build

First thing to do: start GNS3 (which helpfully starts up its own VM.)

Next up: drag all my devices into place, and connect them.

Almost immediately I hit a snag: it turns out you can only have one device connected to a cloud interface. So I ended adding another router to the mix. At the end, it looked like this:

Next step was an IP addressing scheme. It's really complicated and not that exciting, so I'm going to skip it. Now for the fun part: going in and manually configuring all those routers.

Immediately, another problem emerges: the 2691 is a layer 2 switch. All the switching ports I used to set up the mesh can't be assigned IP addresses.

After some digging, I found that I should be using a Cisco 3745 router instead. Rather than using the 16-port switching plug-in, I just added a single extra virtual Ethernet port/card for each connection.

(Several hours later...)

Finally got it done.

With 7 out of 8 virtual routers running, as well as the Win2016 VM, my physical machine's barely using any CPU. (RAM is another story, but to be fair, I also have about 20 other programs open.)

Next step's going to be configuring all the routers to talk to each other... but that's for another day.