14 July 2018

Day 2: Building an outpost with DHCP and DNS.

Previously, on Simulcron...

I'm giving up on the VPN thing for now. I suspect my "double NAT" setup is causing problems for IPSec. (Also, I just checked, and for some reason Windows is back to forwarding packets 

Anyways. Thus far on my Win2016 VM, I had ActiveDirectory services and DNS installed. I set up DNS under the "outpost.amateria" domain ("Amateria" is an age from Myst III that I like.)

Next order of business was to set up a DHCP server so I can hand out IP addresses to the VMs on my VMWare LAN.

Oddly, when I went to configure the DHCP server, it failed out with the error "The DHCP service could not contact Active Directory." This error was a little misleading - after doing some digging, I discovered that I'd forgotten to add my user account (that I was using to add DHCP to AD) to the Domain Admins account and that I actually just didn't have permissions. Added myself to Domain Admins and rebooted, and the problem solved itself.

Next step was to actually configure a DHCP scope. The only options I really cared about were the Router (default gateway) and DNS Server options.

With all this done, I decided to go ahead and add a Windows 8 VM to my new domain. Done, and done. I tested and confirmed my new VM has access to the Internet. (I also tested to see if it could send traffic to my VPC. No dice.)

Undecided what to do next...

13 July 2018

Day 1.2: VPN troubleshooting

Starting point

This entry has been a couple days in the works. Mainly because I didn't want to be writing up everything I was doing while I was also trying to Google and troubleshoot.

So after all my hard work trying to set up a VPN tunnel, I realized that my local network at home (where my virtual W2016 server had its "external" IP) used part of the same IP range (10.0.0.0/24) as my VPC (10.0.0.0/16).

Moving ahead

I took the easy way out and changed my home network to use a 172.* private address range instead.

Unfortunately, my VPN tunnels still weren't working. EC2 showed them both down. Tracert'ing shows that my VM was trying to route VPC-destined packets into my local network instead of over the VPN tunnel.

I wasn't totally sure what I did last night, so I decided to wipe out RRAS and related roles on W2016 and start from scratch.

Didn't help either. At this point I realized I didn't know nearly enough about IPSec to understand what I was doing, so I put it on hold.

By a lucky coincidence, I happened to be reading about IPSec in my CCNA study guide yesterday, which got me at least a basic understanding of what I was actually trying to do.

Armed with this, I gave it another shot last night.

First realization: I have a NAT router between my home network and the Internet. This is a problem since IPSec is stateless at the network/transport layers. I added port forwarding to the router to make sure the IPSec packets were able to make it back to the VM.

Still no dice. For reference, here's what my setup at home looks like right now:

Don't hate - I made this in PowerPoint, from memory, at work.

So I started running Wireshark on my host machine to watch for packets going to the AWS gateway.

I discovered that packets were making it to the local LAN, but with a source address on the 10.* network. It was at this point that I realized that I never set up NAT on the Win2016 VM like I'd intended (which would have made the Win2016 VM the single entry/exit point for traffic going in/out of the VPN) - so I went ahead and did that. I also set up a static route on the VM to route traffic destined for 10.0.0.0/16 to itself.

Finally, after watching Event Viewer and the Windows Firewall monitor, it looked like I was having some limited success! Windows was negotiating with the VPN gateway and setting up the tunnel successfully.

Unfortunately, watching Wireshark... I noticed that after the VPN tunnel was established, all the data packets headed for the VPN got an ICMP "Port Unreachable" response.

At this point, I found myself wondering if AWS was returning that response as a "translation" that something was misconfigured on the other side of the VPN gateway. (The port shouldn't have been actually unreachable since that's how the tunnel was set up.)

Minor correction to the graphic: at this point, I still had my VM LAN set up as 10.0.2.0/24. I figured I should probably change this to make sure I wasn't having more subnetting issues, so I switched the VM LAN over to 10.1.0.0/16.

For some reason, at this point, the IPsec rule stopped working. I called it a night since it was past midnight.

That's where I'm at now. More to come when I get home...

10 July 2018

Day 1.1: Implementing a VPN connection to an "outpost."

Getting Started

VMWare Setup

First thing I need to do, like I described in the design, is set up VMWare with a "virtual network" and then give one VM the ability to talk to the outside world.

Added a "host-only" network, 10.0.2.0/24.
My W2016 VM was already built, so now I just needed to configure it as a go-between:

VM configured with two network adapters: one on the virtual network I just created, the other sharing an IP address with my physical machine.
Once I started up the W2016 VM, it was a little confused, since its configuration wasn't what it was last time it was powered on.

I renamed the connections to match their respective "sides." One of them isn't working, I'm not sure why. I'll tackle configuring the INTERNAL side first.
I've decided my "outpost" subnet is going to be 10.0.2.0/24 and my "hub" is going to be 10.0.2.100. There's no default gateway yet. For now, I'm going to have it use Comcast's DNS servers.

Now to tackle the other problem: Google suggests that this is a problem with using NAT for that interface, so I switched it over to "Bridged." Problem solved.

Setting up the VPC for VPN (!)

I basically just followed this guide: https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/SetUpVPNConnections.html to set up the VPN connection itself (using static IP addressing.)

---

Okay, after all this, I found a major flaw in my plan.

My VPC address range is 10.0.0.0/16.

My internal network here at home is 10.0.0.0/24.

The VPN tunnels aren't working because W2016 is trying to forward traffic destined for the VPC, out over my local network instead of through the tunnel.

I think I'm gonna work on this some more tomorrow.

Day 1: Designing a VPN connection to a VMWare "outpost."

Starting point

As of this point, I have a functioning Windows 2016 domain controller and DNS server, which is the starting point for the rest of the network.

Next objective

I have a VMWare Workstation Professional install running on my local machine with a bunch of workstation-type VMs installed inside. As far as testing out my domain's abilities to centrally manage a bunch of different systems, I'd like to be able to use these VMs - because EC2 instances mostly run server operating systems.

One of my local VMs is a Windows 2016 Server, which is perfect! It can serve as a local domain controller for my VMs.

Of course, there's one problem: my VMWare install is running here, and my primary domain controller is running in AWS.

So that's today's problem: establishing a VPN tunnel between my W2016 VM, and my VPC running on AWS.

(This is an unusual scenario that most people won't need or want to do, but I want to, so...)

Design

There's a few problems I need to overcome:
  1. My local machine is on a Comcast residential connection, and can't be relied on to have a static external IP address. 
  2. My W2016 server is running inside a virtual machine, on a physical machine, which is behind a NAT firewall.
The solution to #1 is pretty straightforward: it means the VPN tunnel will have to be initiated from my end, every time.

#2 is more complicated. I don't want my VMs to be on my local network, I want them isolated in their own virtual network. But I also need them to be able to talk through the VPN.

The solution: VMWare will allow me to create a virtual network with no direct connectivity to the outside world. All the VMs will go on this network. The W2016 VM, though, will get a second virtual NIC that has connectivity to my local network, and from there to the outside world - through which it can establish a VPN tunnel.

I'm not certain at this point whether there will be any ill effects from maintaining two separate domain controllers that don't stay on 24/7. But we'll find out.

Day 0.3: Building my domain

(NB: I'm posting this out of order since I was "live-blogging" as I went tonight, I need to go back tomorrow and explain what I already did in AWS to get to this point.)

Starting point

So far, I've gotten my VPC and routing configured in AWS, and created an EC2 instance running Windows 2016 Server. I've run Windows Update to bring the server up to date on patches and renamed the server to STARBASE-HUB. I've installed the DNS and ActiveDirectory Directory Services roles on STARBASE-HUB.

Doing work 

Alright, so... the first thing I wanted to do was set up an ActiveDirectory domain to link together my virtual WAN.

I own the domain pyhakon.net. ActiveDirectory is based off of a DNS structure - you don't HAVE to give it a valid domain name to work off of, but since I have one, I figured I might as well. Making a Windows server a domain controller requires that it also be a DNS server, so it made sense to make that domain controller the "root" of my sandbox network. I made my sandbox domain "starbase.pyhakon.net" because I'm a nerd. Accordingly, since this server was going to be the "nexus" of my kingdom, I named it STARBASE-HUB.

I already use AWS Route 53 to handle DNS for pyhakon.net, so all I had to do was add an "A" record for starbase.pyhakon.net pointing to the Elastic IP I have assigned to my DC/DNS server, then delegated authority with an "NS" record for the subdomain `starbase.pyhakon.net` to `starbase.pyhakon.net`. Anything under starbase.pyhakon.net, will now get referred to starbase.pyhakon.net itself for resolution. (I'm not really touching on Route 53 because it's not part of this setup.)

Next order of business was to set up the DNS server on STARBASE-HUB. This means opening up the DNS Manager from the Tools menu of Server Manager.

Right now I just want to set up a basic Primary Forward Lookup Zone for starbase.pyhakon.net. Accepted pretty much all the defaults. After that was done, I added an "A" record for starbase.pyhakon.net and for starbase-hub.starbase.pyhakon.net.

DNS manager showing starbase zone setup, and an nslookup query confirming the host resolves.

I went into Server Manager, which was complaining that I never configured the server as a DC after installing ActiveDirectory.

Next I got asked what I was trying to do with my domain controller: add it to an existing domain, create a new domain within a forest, or a new forest. Simple answer: domains belong to forests, and if you don't have an existing forest, you have to create one.

Next I got asked about the functional level of the new forest/domain and a few other things. Of note is that I can't make this a read-only domain controller because you can only do that if you're adding to a domain where a normal domain controller already exists.

Next I got a warning that it couldn't create a delegation upstream for starbase.pyhakon.net, because the upstream DNS server is not Windows, it's Route 53. This is fine - I already did that delegation manually.

It asked me if I was okay with "STARBASE" as the NetBIOS domain name. Sure.

Asked me to set up a few paths. Defaults were fine.

Review options, and... go.

It pointed out a few warnings, mainly that I should disable "Allow cryptography algorithms compatible with Windows NT 4.0" for security, that I didn't have a static IP address assigned to the machine (which is fine - EC2 will make sure I always get the same IP address) and the previously mentioned DNS error.

Happy with itself, the server rebooted. When I reconnected, RDP complained that it no longer recognized the server (which now identified itself as starbase-hub.starbase.pyhakon.net.) I was okay with this, but now had to wait for the Group Policy Client to set itself up... which takes forever the first time.

Cool. I now have my own ActiveDirectory domain.

What's Next

Next I want to make a new domain within the forest, which is going to control all my VMs running on my local machine. This is going to take some design work, which I'm going to do tomorrow since it's 10:15 now.


Day 0.2: AWS Design/Setup

Getting Started

I'm writing this section retroactively, since I already did most of this before starting this blog.

Building a VPC

Intro

VPC - Virtual Private Clouds - are basically private networks (existing inside the standard private class A/B/C IP ranges) inside of which you can launch EC2 instances. You can subnet VPCs, create custom routes and ACLs between them - pretty much anything you could do with a physical network.

I decided to use 10.0.0.0/16 as my private network to give myself room to subnet (and because I knew if I went with class B, I'd never be able to remember which 172.* network I settled on.)

When I did this initially, I created a single subnet that took up the entire /16 address space of my network and started putting machines in there.

This was a mistake. Afterwards, I realized I wanted to make actual /24 subnets, but at this point I already had EC2 instances assigned to the /16 subnet and couldn't figure out how to get rid of it... so at this point I just gave up and started over from scratch with a new VPC. You live, you learn. Fortunately, I hadn't done much at this point.

Final(?) Design

I settled on 10.0.0.0/16 as my VPC network, with a single subnet of 10.0.0.0/24 to start. Since I was going to start out building a domain controller, by default, 10.0.0.0/24 is now my "core" subnet where critical servers are going to live.

Big takeaway: don't create a subnet that takes up your entire VPC if you want to be able to create smaller subnets later on.

Implementation

VPC Creation

First thing to do is create the actual VPC. This is pretty simple - all you need is the CIDR range of the network (though it's helpful to give it a name, too.) You can also specify that you want dedicated hardware for it, but I don't need or care about that. I also don't care about IPv6, at least not right now.

New VPC: 10.0.0.0/24


Most of the default settings (ACL, routing table, DHCP options) for a new VPC are perfectly fine, with one exception: new VPCs have no "Internet Gateway," which means they have no way to reach the Internet. (That might be okay if you only want this VPC to be reachable from other VPCs, but for me, that just means I can't reach it to set it up.)

Adding a new Internet gateway is easy. Like, really easy. You go to the Internet Gateways section of the VPC console and create one. You don't even have to give it a name (though you probably should.)

Route Setup

The easiest way to get at your VPC's routing table is to open up your VPC listing screen, select your VPC, and click the routing table in the window at the bottom.

Select the routing table from the list that appears, and in the bottom window, select "Routes." By default, you'll only have one route, for your VPC's CIDR address with a target of "local." Select "Edit," and add a new route for destination 0.0.0.0/0, with a target of the Internet gateway you selected in the previous step.

Routing table with 0.0.0.0/0 route added to my Internet Gateway.

Adding a Subnet

One last step: creating a subnet within your VPC. Add a subnet from the Subnets category of the VPC console. You can (should) specify a name, but you must specify the VPC you want to add the subnet to and the CIDR address of the subnet itself. Availability zone doesn't matter - that's for people wanting to design redundant EC2 systems.

Again, I don't recommend creating a subnet that takes up your entire VPC unless you know for sure you're not going to want to create any other subnets later on.

Subnet 10.0.0.0/24.


You now have an Internet-connected VPC with a subnet, ready for EC2 instances.

Creating EC2 Instances

Launching an instance

The first step is to build the hardware requirements for the instance and then save it to create the instance.

From the EC2 console, select Instances, and Launch an Instance. You'll first be prompted to select the operating system configuration to install on the instance. There are a whole bunch of operating system options, with various customizations. For my initial setup, I only needed a basic Windows 2016 Server setup.

The next important consideration is what type of instance you're going to run; they come in various configurations, some specialized for memory, some for storage, some for computing power. I just needed a generic instance, with enough memory to run W2016, so I opted for t2.micro.

On the next screen, there are a couple important options: Network, where you specify the VPC you want the instance in, and Subnet, where you specify the subnet within that VPC. Slightly further down, under Network Interfaces, you have the option of specifying the primary IP address that will be assigned to that instance.

On the next screen, you specify how much storage space you want the instance to have. Different operating systems/configurations have different minimums; the minimum for a W2016 Server Base is 30GB.

On the next screen, you can add your own "tags" to the instance. Don't need to here.

The next screen is important. This is where you can specify the firewall rules that will apply to your instance right away. Since the Internet is full of infected systems automatically scanning for other systems to infect, you want to minimize the chance that your system will get infected before you've had a chance to harden it. With that, two things to keep in mind. First, out of the box, you should only allow a connection to the service you're going to be using to configure the machine (typically RDP for Windows and SSH for Linux.) Second, for that service, you should (at least initially) restrict access to the IP address or subnet that you're going to be administering the machine from. I opted to use the /16 of my own IP address just in case it changed, which still restricts access to a relatively small subset of random Comcast users.

Security group for my "hub" server.
(Note: this screenshot was taken after I'd already patched the machine and set up a DNS server.)
Summary description of my hub EC2 instance.
(Note: Elastic IP is already assigned.)

Adding a Public IP

AWS assigns public IPs with something called "Elastic IPs."

The idea is, you get assigned a public IP address from Amazon's pool, which you can then assign and swap between your EC2 instances at will (though you probably don't want to do this.)

Since IPv4 addresses are a limited commodity, and they can't be reused as long as they're assigned to your account, Amazon will charge you (not a lot, but they will) any time they're not being used (i.e. attached to a running EC2 instance.)

Creating Elastic IPs is easy, though. Go to the Elastic IP screen of the EC2 console, and allocate a new address. Next, right-click the address, and choose "Associate Address" to tie the Elastic IP to the private IP address of an EC2 instance.

You can verify the configuration by looking at the configuration of the Network Interface attached to the EC2 instance.

Network interface with a public IP attached.
(Note: this instance has two "private" interfaces. The primary is attached to the Elastic IP and is publicly accessible - the other is not and can only be accessed from the inside.)

Summary

So far, we've:
  • Created a VPC
  • Added an Internet gateway to the VPC, and made it the default route for non-VPC destined traffic
  • Created a subnet within the VPC
  • Created an EC2 instance
  • Restricted inbound access to the EC2 instance
  • Created a public IP address and attached it to the instance.
Next post is setting up the instance as a domain controller.

09 July 2018

Day 0.1: Party Supplies

This is going to not be as detailed as future entries hopefully will be, because I actually did some of the work already... but I'm currently at a standstill because my local W2016 VM has been chugging away at a Windows Update for the last half hour, and I burned through all my CPU credits on my AWS EC2 instance, so, I figured I'd try and explain what I used to get started.

What You'll Need

Thanks to the magic of virtualization, you no longer need to fork over hundreds-to-thousands of dollars to have access to a physical lab to play around in. You have a handful of options available to you instead (this isn't an exhaustive list, it's just the ones I know of):

Local Simulation

There are a handful of options that allow you to set up and run virtual machines on your local machine. The significant drawback to this is that running VMs can be very resource-intensive; there is only so much you can do before your host machine slows to a crawl. Another is that if you want to play around with Windows Server... you're going to need a license for Windows Server, which is not cheap. (I'm fortunate enough that my grad program gives me free access to year-long licenses for certain software programs, so that what I'm using for my local W2016 setup.)

VMWare

Probably the best-known virtualization software for virtual systems is VMware. VMware comes in both free and paid editions. The free edition allows you to set up a virtual machine fast and quickly; the downside is that you're very limited in what you can do as far as building multiple VMs and connecting them together. The professional edition (pictured below) is much more capable and lets you build virtual networks of VMs, and things like that... but again, it costs money.

GNS3

GNS3 is a free program that lets you build networks of virtual network devices and link them together. There's no paid version, and it's extremely flexible. The downside is that it can be somewhat complicated to set up - and even more significantly, the virtual devices require a ROM image of the (typically Cisco) device you're trying to emulate, and most people don't have legal access to those. So I'm not gonna touch on that one very much.

Cloud-based Simulation with AWS

This is where, in my opinion, you have the most flexibility to explore, at the lowest cost. Amazon's AWS cloud platform offers you a stunning amount of virtualization capability, for extremely cheap. There's so many offerings that I don't even know about most of them; but luckily, for building sysadmin sandboxes, you mainly only care about Elastic Compute Cloud (EC2) and Virtual Private Cloud (VPC.) I'll try and sum up how it works really simply.

When you build an EC2 "machine," you specify a bunch of configuration parameters called an "instance," and create a virtual hard drive for it, both of which get stored in the cloud. When you start that instance, Amazon grabs your configuration and launches a virtual machine from its massive bank of computing power that matches those specifications. (Unless you ask, and pay extra for it, your instance will probably never run on the same physical hardware twice.) It attaches your virtual hard drive to it, attaches "your" IP address to it (more on that later) and you're good to go. The entire process is completely transparent to you - as far as you can tell, any time you connect to that IP address, you're always operating the same system. And it's really, really cheap - they actually offer a "free" tier of service for the first year of having an account, where if you use the service very little, you may actually end up not having to pay anything at all. If you don't (and you probably won't, if you want to do anything more advance) it's still ridiculously cheap. As of this writing, my cost for a single Windows 2016 EC2 instance operating in the "micro" tier (1 CPU, 1 GB of memory) is:
  • Runtime: $0.0162 per hour that the instance is running. (You can turn it off when you're not using it!)
  • Virtual hard drive: $3 per month (30 GB x $.1 per GB per month.)
  • Public IP address: $.01 per hour that the instance is not running. (IP addresses are reserved for you - Amazon doesn't mind letting you have it for free as long as you're using it, but if you're not, then they're going to charge you. Unlike EC2 computing power, they can't just assign it to someone else - nobody else can use it as long as it's assigned to your account.)

VPC is similar. A VPC is basically a private network that you can divide and configure as you please. (This is great for understanding the fundamentals of routing concepts - but as far as Cisco certifications go, since all the configuration is done through the AWS console, you won't learn the Cisco command-line skills you'll need in order to pass Cisco exams.)

What's next

Since AWS is the most flexible and scalable, I'm going to do the majority of my work in there. I do have a local Windows 2016 VM that I also run (along with some desktop VMs) that I plan to link up with the AWS instances, just because, but that's more complicated than most people will want to do.

It's 9PM and I may not get to the rest tonight, but next entry will cover what I've done so far.

Day 0: Makin' a Blog

Recently I've set a goal for myself of getting certifications to formalize a lot of the stuff I've learned over the years, and pick up some new stuff. I've never liked to specialize when it comes to computers, so my goals are kind of all over the place: MCSA, RHCSA, CCNA, and so on.

The problem I've found is that most study books focus on teaching concepts, and only teach you how to "follow the steps" to accomplish a single task at a time. This is not great when it comes to learning by doing, nor does it let you see all the pieces working together.

Which got me wondering if there was any kind of repository out there, of scenarios that give you a network/system configuration to build, and let you figure out how to make all the pieces work together - since thanks to the magic of virtualization, the days of needing a complex physical lab setup are over!

I didn't find one.

Undeterred, I figured I'd try and use my imagination to conceptualize my own network and build and expand it.

I have, like, a 30 minute commute to/from work (Schriever AFB, holler!) and a whole lot of free time while I'm at work since I'm in the process of getting out, which gives me a lot of time to think about things. On the way home today, I realized I should blog my experiences, because they might help some people recreate my adventures and make some of their own from there. Also, some day, I might turn them into the "scenario bank" I couldn't seem to find.

"Let the games... begin." - Victor Nefarius, Lord of Blackrock Mountain

I fought FreeRADIUS and FreeRADIUS won

Back to work So...  last I had written, several months ago,  I had managed to add my CentOS server to my domain and was going to work on ...