Help with SOHO "data center"

I have building an application platform, and plan to deploy it on Google Cloud (GCP) once it becomes a viable product and makes money. ;-)

In the interim, I would like to setup my home network to be able to run the same platform as the one that will eventually run on GCP.

The question I have is not about if I should just use CGP, another Cloud provider, etc.

Lets take it for granted that it is a good idea to run this from my own infrastructure at home.

Here is a basic diagram of as is network at my soho network.



Like most home networks, I have a home ISP service with dynamic IP, and where I do all my development, and a NAS (Unraid Server), and a bunch of VMs that allow me to create a dev\test infrastructure.

I also have a HomeOffice ISP service with Static IP, where I would like to put all my "Production" servers for my platform\product.

My needs are:

1. What approach/setup do I use to deploy/connect my dev components to the production network
2. What firewall setup would you recommend (right now only ones are the one in those cheap routers provided by the IPSs)
3. Is there a setup that I could use to locally connect the production network to the dev/home network?

200+ views, no opinions?

Please, help... Having to learn so many technologies lately, I think one more will make me insane!

I'm not sure I'll be a lot of help, but I'll take a couple swings to get you started. It sounds like you kind of want to start from scratch. If you have anyone else in the household, or you just want to maintain your sanity, I would seriously consider keeping your home network fairly simple and completely separate from your dev/test/lab network. That way you can break things to your heart's content and not worry about broken home networks. Same with your business network, if you have workstations and stuff that need to be reliable, I'd keep them separate--basically the same as your home network, just on their own VLAN. Then everything that wants to get wild can be on its own VLAN/subnet and that's where you can break it all, blow it up, catch it on fire, etc.

I would get off the basic ISP modem/routers, and go with one device. You show two in the image, unless your business line is totally separate--even then you should be able to combine it into a single device. There's all kinds of options: Ubiquiti, Mikrotik, PC-based distribution (or PC-Engines based), the sky is the limit. Ubiquiti stuff is OK, I run it at home and it's pretty reliable, but it's not the enterprise-grade stuff the "rabid" fans tout it as. If you want one device to do all the routing/firewalling for two connections, you're probably going to have to go up to something like an ER-4, USG Pro, or something PC-based. But if this is for a profitable business, I'd also look at some of the more business-class stuff as well (Merakai, Fortinet, etc).

So, in your business network, do you have development and production? From your diagram, is your business line specifically for hosting? Do you share this network with others in the household?

My home network is divided into DMZ/internal/guest/etc, so based on my experience I would suggest something like this:

-New firewall/router that can handle your dual WAN.

Internal Network/Home ISP:
-Internal VLAN for trusted devices, NAS, workstations, etc
-Internal VLAN for guest devices, IOT, etc (primarily wifi)
-Internal VLAN for testing/developing systems. This way you can break/destroy everything, something can get loose and cause chaos, and it's all limited to that particular subnet
Business ISP:
-Flat network for your production servers only

I would set up the router/firewall with ACLs to allow traffic back and forth, but being very careful to only allow it specific ways. Allow your dev workstation to access machines in the dev and business network, but don't let those devices into your trusted VLAN. I would consider carefully where some servers and systems might go. A NAS that's used to store photos and media should stay on the home network and not be involved with the business systems. The dev database server and file storage should stay in the testing section, etc.

I don't know if this will get you started, or it's what you're looking for, but there's a few ideas.

Sorry I disagree with bridging these networks at all. Keep it with two separate ISPs, don't connect them at all.

Your original post described basically selling this app/service until it makes enough to move it to the cloud.

First and foremost, it's going to be tough to just 'move' it to the cloud if it isn't designed the right way in the first place, unless you just want to run VM's on google and migrate your home vm's to it. I'd seriously rethink how you want this to grow and transition

^^ That, plus why ever bridge them anyway? It's not going to be a trivial task, and doing it would essentially break any form of security you have. This is a bad idea.

Is it unreasonable to have two separate networks with a single router/firewall? You can separate all of the traffic and keep them on their respective links but still use a single device, and allow a connection through from the internal network for access without going out through the internet and back. It's not a simple configuration, but from a security standpoint it's not much different than a DMZ right? The big issue is effing up the configuration and allowing something in that shouldn't be--so from a configuration standpoint two devices two lines etc etc would be less prone to messing it up.

I do agree it's a bad idea all around to be hosting stuff from home, but was assuming this was mostly for development. You could probably even get a server in a colo that would be a better solution for the same price as the business line at home and it would be far more appropriate.

ETA: Nevermind, you're right especially for someone that's learning all this stuff at once that's going to be a bit too complicated to learn off the bat. I'd just get a colo server and keep the home network as the home and test lab and connect to the remote server/host/etc.

I'd agree with Eickst that if your end game goal is to get everything onto GCP, you probably want to model it around how you'd build the infrastructure on GCP. For me I think it's a pretty hard sell to not just spin up the web server in GCP because it's going to be a lot less painless.

But obviously to your point you want to build it yourself. Which is fine, and I can at least give you my take on how I'd build it if I were doing it.

Here's how I would do it.

Business ISP -> Hardware Firewall (PFSense) -> switch -> VMWare ESXi -> VMs.

^
|

Internet


^
|

Soho Appliance <- switch <- Dev pc using OpenVPN


So keep both of the networks 100% separate, but then install a VPN client onto your development computers so they can connect into your hardware firewall. This connection will be going over the internet at all times, but this is exactly how you'll be using this once you go to GCP anyway. (You're not going to be able to just plug your LAN into GCP)

I much prefer a hardware firewall over a VM, but that's up to you. You certainly could host your Business ISP firewall as a VM on your production VM server. My main issue with that type of setup is that when things go wrong, they tend to go very wrong. It can be much harder to get everything unbroken if the basic networking is destroyed.



The Web and DB VMs should be on separate networks, so you at least have the ability to firewall between the two. So more specifically I'd do something like this:

Business IP
100.x.x.x -> Firewall -> switch networks

Firewall networks:

192.168.1.x / 24 = Management (VM hosts, switch default network, firewall management IP, etc)
192.168.2.x / 24 = VPN Network (The network your dev PC will land on when you VPN in)
192.168.3.x / 24 = Web Servers
192.168.4.x / 24 = DB servers


Then make is so that 192.168.2.x can connect to the other networks, but then make it so that none of those other networks can talk to each other. The finally you would add in a rule that allows the web server to connect to the DB server on whatever port it needs. That would satisfy the basic networking behind the firewall. You could use VLANs to a managed switch, or you could just use a computer with a couple of NICs in it. There is only 4 connections that would need to physical, so a pair of dual port cards would do everything you need. (The VPN is a virtual NIC, but you need to have a port for the ISP connection)

In your case I'd make it as simple as possible. The 192.168.1.x network in my example could be connected to a dumb switch. Then to make the connections for the VM server to the firewall, you just need a pair of cables to connect them directly together. No need to worry about tagged traffic, or learn any high end networking. You'd just configure those bridges on the VM server and the virtual switches on that VM host would allow you to put multiple VMs onto the same network. So a crude diagram would be something like this:

Firewall PC

NIC Port 1: To ISP device (Modem, switch, etc)
NIC Port 2: LAN Port / management port connected to dumb switch (192.168.1.x)
NIC Port 3: Web DMZ to VM host (192.168.3.x)
NIC Port 4: DB DMZ to VM host (192.168.4.x)


VM Host

NIC Port 1: LAN Port / management port connected to dumb switch (192.168.1.x)
NIC Port 2: Web DMZ connected directly over a cable to NIC Port 3 on Firewall PC (192.168.3.x network, VM Host does NOT need an IP for this to work)
NIC Port 3: DB DMZ connected directly over a cable to NIC Port 4 on Firewall PC (192.168.4.x network, VM Host does NOT need an IP for this to work)


A couple more points, you'll still need to add in a forwarding rule so that your web server can be reached from the internet. So basically you need a port forwarding rule for 100.x.x.x:443 -> 192.168.3.x:443 (whatever IP you end up choosing) That is what will actually put the server online. The reason of choice for using VMs on something like VMWare is easy enough because there is the ability to migrate those VMs directly to the cloud. https://cloud.google.com/compute/docs/vm-migration/using-velostrata (Not necessarily saying that I'd go that route, but it's certainly an option so you don't have to rebuild the entire thing right away)


You'd definitely need a couple extra pieces of equipment if you don't have them for my suggestion to work. Not sure what you were planning on doing there but obviously I'd never run my dev stuff on the same equipment as my prod, so the unRAID Nas is out. You could use unRAID instead of VMware, but I can't say that I would suggest that either. The migration path could be a bit more tricky to the cloud, but the bigger issue is that unRAID is not designed with security in mind. It's swiss cheese at best out of the box, so you'd NEVER want anything on the internet pointed at anything on that unRAID server until you do a complete overhaul to make it secure. VMWare is pretty decent out of the box, and you shouldn't have to do much for it to be secure. It's also widely supported and the dominant player in the market at this time.

I'd also point out as one last little snippet of advice, if you're really wanting to build a scaling web app, building it on a VM and then forklifting that VM to the cloud is not the right way to do it

Since your plans are to move the production side at some point in the future, everything you setup/buy for the home side should work with both your temporary business setup and with Google Cloud. That way, when the great day comes and the business side goes cloud, your home setup will just need to be pointed to the Google IP address.

To that end, don't combine anything. Combined equipment will make the move more difficult. From a security standpoint, get firewall/router appliances that support VPN with one end fixed IP and the other end dynamic. Make sure Google Cloud supports the dynamic client. Make sure any equipment you buy for the temp business setup does a close emulation of Google Cloud. Might be worth a chat with Google Cloud support to learn the best way to do what you are planning.

And yes, you do want real firewalls and remember that the first rules to setup are (source IP any, destination IP any, block all services) for both inbound AND outbound. Most ISP and consumer stuff allows all outbound by default. Yes, creating rules for every application and service is a PITA. But so is explaining to customers that your company just joined the long list of hacked companies.

bman212121 said:

I'd agree with Eickst that if your end game goal is to get everything onto GCP, you probably want to model it around how you'd build the infrastructure on GCP. For me I think it's a pretty hard sell to not just spin up the web server in GCP because it's going to be a lot less painless.

But obviously to your point you want to build it yourself. Which is fine, and I can at least give you my take on how I'd build it if I were doing it.

Here's how I would do it.
...
.

Click to expand...

OK, so this is how far I have gotten...

1st - got myself a pfsense box: https://www.amazon.co.uk/gp/product/B07FBZ87NJ/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1

2nd - configured pfSense and connected everthing in the house before adding the DMZs/proxies, etc...
3rd - decided to try out the pfSense Failover/Load balancing feature

Question:
1 - Any issue with letting Home Lan access Swansea ICT LAN and vice versa? Using a Rule? Or shoud use VPN? Main reason to do this, is because I dont want to add another server to the home network for file sharing, media, backups, etc... Basically a SOHO setup ;-)
2 - Should I buy another pfSense box for redundancy?
3 - Will the "firewall" functionality on those ISP modem\routers boxes mess up or interfere with pfSense? I could turn them off, but maybe not on the HOME LAN side?

I will be adding the servers next, but figured, I would ask if this makes sense so far:

1 - Any issue with letting Home Lan access Swansea ICT LAN and vice versa? Using a Rule? Or shoud use VPN? Main reason to do this, is because I dont want to add another server to the home network for file sharing, media, backups, etc... Basically a SOHO setup ;-)

If you must, I'd setup firewall rules to only allow home lan to initiate/establish the connection first and don't allow the ICT lan to talk to home lan.

2 - Should I buy another pfSense box for redundancy?

Wouldn't it be cheaper just go push this to google cloud instead?

3 - Will the "firewall" functionality on those ISP modem\routers boxes mess up or interfere with pfSense? I could turn them off, but maybe not on the HOME LAN side?

I'd disable the firewall functionality on the ISP modem/routers and let PFsense do its job. I'd also take your home lan switch and place it behind the pfsense firewall.

Shockey said:

1 - Any issue with letting Home Lan access Swansea ICT LAN and vice versa? Using a Rule? Or shoud use VPN? Main reason to do this, is because I dont want to add another server to the home network for file sharing, media, backups, etc... Basically a SOHO setup ;-)

If you must, I'd setup firewall rules to only allow home lan to initiate/establish the connection first and don't allow the ICT lan to talk to home lan.

Click to expand...

Cool, I'll do that!

2 - Should I buy another pfSense box for redundancy?

Wouldn't it be cheaper just go push this to google cloud instead?

Click to expand...

Whats the fun in that ;-) This has been a great learning experience. Well worth the £255.00 to learn some more.

After all the money spent so far or gaming hardware, it would be a small price to pay for no single point of failure on the network.

3 - Will the "firewall" functionality on those ISP modem\routers boxes mess up or interfere with pfSense? I could turn them off, but maybe not on the HOME LAN side?

I'd disable the firewall functionality on the ISP modem/routers and let PFsense do its job. I'd also take your home lan switch and place it behind the pfsense firewall.

Click to expand...

The reason I kept the Home Lan outside of the firewall, was not to piss off the kid gamers with extra latency.

But I hear ya. Maybe they won't notice, specially if I turn off the ISP firewall ;-) agree?

SpeedyVV said:

Cool, I'll do that!



Whats the fun in that ;-) This has been a great learning experience. Well worth the £255.00 to learn some more.

After all the money spent so far or gaming hardware, it would be a small price to pay for no single point of failure on the network.



The reason I kept the Home Lan outside of the firewall, was not to piss off the kid gamers with extra latency.

But I hear ya. Maybe they won't notice, specially if I turn off the ISP firewall ;-) agree?

Click to expand...

If you leave ISP firwall enabled you are double NAT'd. That not a good idea.

I thought Firewall and NAT are two different things .....

Shockey said:

If you leave ISP firwall enabled you are double NAT'd. That not a good idea.

Click to expand...

tedych said:

I thought Firewall and NAT are two different things .....

Click to expand...

Now I'm worried.

I know what NAT offers, but how it works, I know exactly zero!

Shit, I guess something else to learn.

Firewalls can do NAT, or they can do tap/inline, most soho firewalls act as your main router and do nat

It's just harder to manage because if you need to set static addresses or do port forwarding you'll have to do it in two places.....