beast home "Core" Build -- theory

X

XCLR82XTC

Limp Gawd
Joined
Dec 23, 2009
Messages
315
So i happen to think that with the advent of 10gb ethernet, and fiber becoming more prevalent that more applications will take advantage of working in the cloud. I work with 3d CAD and they are already starting to move processing to a central unit. If that happens.. then i expect fully the majority of people will be returning to cheap, low power terminal stations/thin clients. Personally I think this is great as long as its secure.


That said, I would like to build a home "Core" I want to build something with a ton of power, that over the few years can be upgraded to lastest components.. something that will run multiple terminals, run the smart TV and act as a home security system monitor, and control automation devices (lights, power outlets etc) pretty much all at the same time.


Now i think i should start with the wiring. I dont think the hardware/software is there yet to do what im picturing in my mind, but by the time I get all the wiring in, and a server room in place.. MAYBE hardware will have enough cahoonies to future proof until cloud apps really start coming down.


IF you were going to take on a project like this, how would you go about it? Im thinking I should make Fiber runs to each room. Build in boxes to contain wireless AC routers and go ahead and get the server room built with full climate controls. I think the next step should by buying security system components (cameras, sensors etc ..that will probably need somthing more like a plc to control honestly) Then move on to automated light switches and power outlets. Surely by then the hardware should have kicked it up a notch. Im looking to make this a 2-3 year project before I even start on building the actual server.

Id also like to think i could make this modular.. something I could easily add memory to, or even extra processing power via video cards.. or somthing like that..


should i wait till 2020?
 
"That said, I would like to build a home "Core" I want to build something with a ton of power, that over the few years can be upgraded to lastest components.. something that will run multiple terminals, run the smart TV and act as a home security system monitor, and control automation devices (lights, power outlets etc) pretty much all at the same time. "

Um... can you program? Do it yourself. Also, I'm pretty sure the technology is available today, it just isn't usually marketed at normal consumers. It's usually marketed to businesses or artists. If you look you will find it.

One of my friends used to work for a company that actually sold light control applications to businesses. Just google "light control software" and weed through the stuff marketed to djs and I'm sure you could find something.

Considering you are thinking of this as a multi-year project it would probably be most rewarding to do it all yourself/write all the software yourself if you can. It isn't that difficult, for the amount of time you are willing to invest in this project. If you don't want to program, just start looking into things like "light control software" and home security systems. At this point, most businesses are smart enough to make an app/desktop software to control their products so it should be there for you.
 
So as for the neworking forum is concerned please share your needs for "networking". not cad software, dj software, light controls,any of that. Just networking.

We will be glad to help bu lets make sure we focus on the networking aspect of your goals.

We need to know things like...

1. I want a FAST nas unit to store data and all that yummy stuff...
2. I want 10gbe and tell us why.
3. im cool with gig but I want top notch performance or I dont care about price as long as its cheap.
4. What cabling tech should I use?

etc...

Again we will be glad to help with the networking related questions but we need a picture painted other than a "Core". Do you intend to build a network rack and if so will it contain rack mounted networking equipment?

Thanks and subscribing to come and offer assistance as needed.

Now so far you mentioned a few things that I was able to ascertain as a need:

1. you mentioned fiber. Why? What is your envisioned goal by using fiber?
2. you keep mentioning the hardware would have kicked up a notch? What are you referring to?

A gigabit switch, gig network cards, and copper lines throughout the house will be fine for probably 10 more years minimum.
A fiber optic setup is going to cost you an absurd amount of money and the only benefit you will get from fiber is much faster latency as far as data transmission is concerned but that is only going to make a difference if the switching hardware is fast enough to utilize the natural speed of light data will be traveling at. Fiber optic switches are also ridiculously expensive if all ports on the switch are fiber. Also fiber will be beneficial if one of your rooms is 10 miles away from the switch. And electromagnetic interference is 100% immune to. Also EM sniffers dont work on fiber i.e. spies listening to your data transmissions by recording the EM pulses generated by your network across copper lines.

If you are planning on building a VMware based server to run a NAS/Server OS/ other custom items and you need to have the added bandwidth and ridiculously over priced and underutilized for 99% of any home user you can consider building for 10gig ethernet. It comes in three forms listed below:

a. Infiniband (cheapest method, bulkiest most obnoxious cabling of the three)
b. 10g/e over copper , uses Category 6 cabling and switches are still VERY high in price. Cabling is also thick but not as obnoxius as inifniband.
c. Fiberoptic, cheap to deploy but the hardware will kick you in your nuts to be able to afford.

Now as far as badass monster switching gear, we can start a conversation about recommendations, and they will include money brands like Juniper/Cisco/Brocade/etc... Get ready for some ebay if you want to save money and I can share very reputable honest vendors I use regularly.

If you want cheap hardware but still good enough to be called enterprise, we will mention brands like HP/Netgear/etc... Okay some HP is very expensive and works but I am not fond of it in the truly enterprise world though it will work and many enterprise companies use tons of HP servers but not so much on the networking hardware side.

If you are talking NAS servers, you have factory pre-built solutions like HP/DELL/Thecus/QNAP/WD/etc...
If you are talking homebuilt, there are RAID based and ZFS based, and the latter I would recommend a zillion times over raid.

the list goes on...Good luck and may the HF community offer much advice.

I have a very nice 10Gbase-SR setup running in my home between a few hosts as well as my switch and NAS server. Absolutely blazing but cost me several thousand to implement and that is using used enterprise hardware that isnt always available on the fleabay.
 
Last edited:
basically what this boils down to is that i want to be able to use a highpowered pc , and whatever software i happen to have installed on it, over thin client, or a remote desktop. not over the net (unless speeds change soon) but inside my hoom, from any room, from any terminal i desire.

I have experimented with VPNs, quad port Nic Cards with teaming enabled, jumbo frames on/off ...nothing really "works" smoothly like being at the actual workstation.

Now you put in all the other things that I want the server to manage...

I think this post might belong more in a general category vs networking, but while we are on networking, is it not true that we are nowhere close to maxing out fiber, and that it should be future proof far more than 10 years? I know the swtich gear is expensive but i would hope in 3-4 years this prices would be down (as were already seeing simple consumer end products on the market reasonably priced)

and for the other poster.. no i dont know how to program I know how to do ladder logic and plc programming, but not comptuer programming. though it is one of the things im going back to school for in the near future. it would definitely help me with so many of my projects.
 
Last edited:
In my experience (I'm no thin client expert) I would never expect great performance from any remote desktop or thin terminal. I may be wrong, but my impression is the protocols aren't really built for multimedia.

Splitting delegation of tasks between different servers or workstations would be a much better idea than trying to cram everything onto one box. Assuming you want to run Windows, relying on a single machine to host everything is a recipe for a massive PITA. Even if you virtualize multiple machines on a single host, you'll likely come across situations where you want to power off for some reason, but don't want to kill your security system or home automation.

If you're thinking of sinking money into a "massive core server" why not get a "reasonable core server" and a reasonable workstation? Why would you need to work from multiple desks in your own house? Also - you will not be able to upgrade past a single generation of CPU without a motherboard (and likely RAM) upgrade as well, at which point there is no practical way to build an "upgradeable" system beyond extra CPU slot and tons of room for RAM.

Also from personal experience, "I want fiber" is something only spoken by people who have just heard about it and/or think it sounds "fancy". For the huge cost difference and installation requirements I can't imagine a home setup that requires fiber.

Or get a workstation-replacement laptop.
 
Innocence said:
Splitting delegation of tasks between different servers or workstations would be a much better idea than trying to cram everything onto one box. Assuming you want to run Windows, relying on a single machine to host everything is a recipe for a massive PITA. Even if you virtualize multiple machines on a single host, you'll likely come across situations where you want to power off for some reason, but don't want to kill your security system or home automation.
...
Or get a workstation-replacement laptop.
Click to expand...

I think you may be missing the point here. He wants to try this at home to mimic where he thinks computing is going. To test out his theories in his own home. Also, many companies are already doing this for large network spaces. It isn't something that is brand new really. And you don't need to shut down these systems for much, you can shut down virtual servers and bring them back up pretty easily, plus it makes it a whole lot more resilient against crashes and server faults, as you can just reload a VM from storage pretty quickly. Also you don't virtualize all the systems on a single host per se. They are based on a system that is comprised of a number of physical servers. The resources of the servers are combined to form what looks like a single host.

Also, the current problem with the systems is the pass through for GPU operations is still pretty weak. I don't think its really as much of a bottleneck on the network side as it is on the general graphics processing and delivery side of the equation.
 
Noother is spot on. lots of software is going the way of server centralized, or distrubuted computing. my personal belief is that one day there will be no large server farms or centrally controlled sites like facebook gmail etc. Nor will people have powerful workstations for each employee. I envision that one day people will have a "core" as i called it and will link to that core for the normal processing. Instead of storing your pictures on facebook servers, each persons cord will run a small program that makes certain pictures public, and displays as a feed as well. No more gmail servers.. each persons server host its own email. i think this is much more efficient and reliable. its easy to blow up one server farm..impossible to break a network with 2 million nodes


I think we will have modular systems that people can add processor or memory modules on to at will (sort of like bolt ons for your car) when i employer needs 5 more CAD stations he will buy a modular core processing unit of xx teraflops and plug it in to the core.. then set up 5 terminals and buy 5 more licenses and he i done.... thats the hardware part thats not there yet, but regardless you still need to infrastructure to transmit the data.. which is why i want to go with fiber now to build for the future..

Im a mechanical engineer by trade. If CAD goes the way of a terminal client with a central processing server (which they are already developming) then i fully expect people to jump on board for the management and energy savings alone. of course i could be talking completely out the side of my mouth and instead they put a chip in each of us that over rives our brains lol.

can you imagine the battery life of a tablet that had to do 0 processing other than basic data transfer and video. with the new technology for wireless, and energy saveing platforms, were talking about 10inch tables with 15 hour life. maybe more. arm/tegra processros are ideally suited for this already.

is it amazing how technology makes circles... terminals and centralized processing was a thing of the past. yet I believe its coming back around, albeit in a better more refined way.

Id also like to help with the freedom box project and intertwine that with my core theory.. this is why i want to get computer programming/engineering skills. I can create you one bad arse machine or new product but computer code overwhelms me.. time to put that arrow in the quiver though.

sorry for off topic....



that whole future theory aside.. i really do want to build a fiber network for the home.
 
Last edited:
i disagree

i think the future is personal computing devices with more and more power... a phone that you use for your whole life...

your employer can load up a desktop on to your phone when you get hired... and you just plug your phone into a dock at work and you get a monitor and keyboard and mouse... the phone can have any number of form factors and any number of interface devices, but it's all streamlined into the same logical use scenario...

take it home and plug it into the dock on your tv and use your own desktop and start watching a stream or something.... (the idea of tv channels is already going away) walk around with it in your pocket while it runs the peripherals in your augmented reality interfaces

most everything is stored on a server in a provider's datacenter... your content becomes aggregated and sold like different products at a grocery store... there is the one store, but in the store the content is sold by hundreds or thousands of providers...

to an extent it's a thinclient setup, but the data wrangling and backup mess is left to the professionals... with fiber simplifying and greatly increasing the speeds of these ISPs networks, i don't see a problem with it... also wireless technologies delivering > 1gbps transfer rates....

ezpz...
 
For what its worth ... Fiber can use what is called DWDM and can hold multiple wavelengths on one single fiber strand (Different color spectrums basically) and have been able to transmit up to 400gbps or more I am sure using this technology. So fiber is in every way superior to copper in all forms of imagination. So for the home user yes fiber is future proof but .... at what point will you have the need to or the resources to actually leverage what fiber can truly do? As we speak my text characters are being sent over fiber from my desktop to my switch to my router and out. I have fiber in my home and I have absolutely no use for it but its damn cool.

I said 1 gbps copper will last atleast 10 more years. Copper will probably not be phased out for a few more decades but once wireless is 100% as reliable and stable as copper and probably multitudes faster it will be the new standard with fiber always reigning on top as the BIG PIPE.
 
Last edited:
I have no idea why this double posted.... please delete.
 
Last edited:
tangoseal said:
For what its worth ... Fiber can use what is called DWDM and can hold multiple wavelengths on one single fiber strand (Different color spectrums basically) and have been able to transmit up to 400gbps or more I am sure using this technology. So fiber is in every way superior to copper in all forms of imagination. So for the home user yes fiber is future proof but .... at what point will you have the need to or the resources to actually leverage what fiber can truly do? As we speak my text characters are being sent over fiber from my desktop to my switch to my router and out. I have fiber in my home and I have absolutely no use for it but its damn cool.

I said 1 gbps copper will last atleast 10 more years. Copper will probably not be phased out for a few more decades but once wireless is 100% as reliable and stable as copper and probably multitudes faster it will be the new standard with fiber always reigning on top as the BIG PIPE.
Click to expand...

If you go with Cat6, you can definitely get 10Gbps, and probably 40 from it on the lenghts applicable to a typical house. Unless you're fairly young and plan to own that home for another 20-30 years, it's tough to see a scenario in which the capacity of copper is exceeded without getting into really expensive equipment.
 
i think im going to go fiber. Wireless AC will take care of what i want for now, and when im done building ill already have fiber in place to hook the (hopeully lower cost) fiber switches to.

I have a feeling in the next 3 years were going to see a massive massive change in compute power. (graphene) and i think everything is going to get heavier and more network intense. Wait till everyone starts demanding full HD 4k 3d Movies instead of what you get from netflix now. even with faster compressions/conversions you still have a ton more data to send down the pipe. Fiber to mee looks liek the only logical choice.

Goodcopper I like that idea you have of a personal computer. I have to disagree slightly though with the way you think its going to be implemented, I think what you talk about will exist but i think all processing will be centralized. especially when it gets to the point that we can send more data than we know what to do with. but hey its the future.. i guess we will see when we get there!
 
Last edited:
XCLR82XTC said:
Noother is spot on. lots of software is going the way of server centralized, or distrubuted computing. my personal belief is that one day there will be no large server farms or centrally controlled sites like facebook gmail etc. Nor will people have powerful workstations for each employee. I envision that one day people will have a "core" as i called it and will link to that core for the normal processing. Instead of storing your pictures on facebook servers, each persons cord will run a small program that makes certain pictures public, and displays as a feed as well. No more gmail servers.. each persons server host its own email. i think this is much more efficient and reliable. its easy to blow up one server farm..impossible to break a network with 2 million nodes

that whole future theory aside.. i really do want to build a fiber network for the home.
Click to expand...
I see - so some sort of nightmare child of Mainframe computing (50 years old, largely abandoned 20 years ago), Cloud computing (5 years old, adoption rate stagnant), Pay-Per-Core Blades (15 years old, low adoption), and Granny's DIY home web hosting (never a good idea, probably never will be)?

Crazier things have happened I suppose. But given this, I don't see how you could come anywhere close to approximating this infrastructure in your home.


Personally I think that we're already seeing the shift that will form the next couple decades of computing. It's very similar to what goodcooper said. Within 5-7 years we'll likely have full desktops in our pocket that will start to replace traditional workstations. Mass storage will be datacenter based for enterprises, and likely cloud based for the majority of home users. CPU power already isn't much of a concern for 90% of users on tablets, and is continuing to grow.
Think Chromebook, but less useless.
 
Actually even Intel is packing more GPU cores into their integrated graphics. Calculations have been moving rapidly to things like graphics cores.

Think about how bitcoin works. As this massive distributed computing thing progresses I feel like there will end up being a system in place where all massive processing work will have a way to be distributed out and rewarding people for their shares. Folding, SETI and something like bitcoin will combine technologies/ideas.

Lets not overlook how much actual power is being packed into tiny devices these days.

My main workstation and my server I still use 1gb instead of 10 and I just have 2 wires to each. Everything else goes by wifi because they just don't matter. So congratulations if you build now you will be spending far more than you need to. My big storage drives aren't fast enough to fill a 10gb pipe so I think you'll be waiting for massive storage to get faster for your pipedream to work. SSD's are in every laptop I have and of course my desktop too. Those just run the local stuff though.
 
I'd wait. 10GbE is too expensive for the home, and Moore's law tells us the cost (not necessarily the price) will halve in 3 years. I have no problem using CAT 5e in homes. If you want to pay the price to lay and terminate Fiber to every drop in your home, and the equipment that goes with it, I have no problem billing you for it.
That being said, Fiber doesn't address any practical (not theoretical or edge case) deficiency in copper. Go ahead and wrap your 5e cable around the compressor in your fridge, then multiple parallel runs next to your fluorescent lights- It will work fine. You'll run into the 300 meter length limitation before you seriously degrade performance with your antics. Copper is going to be around (in the home and office) for a long time, because there is a lot of it around.
Now fiber for transmission and metro distribution has it all over copper- 20+ kilometers between low-powered receivers, the ability to increase transmission speeds by replacing the equipment on either end and not touching the cable, high capacity, moderate-to-low cost, the list goes on. Fiber for transmission and distribution is a foregone conclusion- it will happen. Fiber inside the home and office (for workstation use) is minimum 20 years away and more likely 50. Wireless is the emerging technology for home and small office, much as it pains me to say it.

If the whole purpose of this post is just to stoke your vanity, well just say so and I can give you some great ideas. If you are serious in the contents of your OP- that dog doesn't hunt.
 
Im very serious that I want to build a power home core/data center. and I do fully beleive that more an more applications will be able to leverage the power in the near future. The only "reason" for the fiber is to garuntee 100% beyond a shadow of a doubt, that i wont need to pull it out in 20 years (or 50) as you say and replace it with something else. i could use this "core" RIGHT NOW. without even waiting for hardware/software to evolve. Having 4500 cuda cores would probably cut my simulation times in half.
 
XCLR82XTC said:
Im very serious that I want to build a power home core/data center. and I do fully beleive that more an more applications will be able to leverage the power in the near future. The only "reason" for the fiber is to garuntee 100% beyond a shadow of a doubt, that i wont need to pull it out in 20 years (or 50) as you say and replace it with something else. i could use this "core" RIGHT NOW. without even waiting for hardware/software to evolve. Having 4500 cuda cores would probably cut my simulation times in half.
Click to expand...
...about that. You realize that Fiber cabling standards are still moving with technology, right? If you installed OM2 10 years ago (orange jacket - "the best" in 2003) - you'd still be limited to 10Gb at 300 ft- exactly the same limit as Cat6 copper.

Fibre != Futureproof
 
XCLR82XTC said:
i could use this "core" RIGHT NOW. without even waiting for hardware/software to evolve. Having 4500 cuda cores would probably cut my simulation times in half.
Click to expand...

How would using fiber vs copper RIGHT NOW in the home affect your implementation?. 5e will support 10 GbE for typical home distances. Any equipment you buy now will be deficient if not unusable in 20 years. Save your money. Even 1GbE will support multiple HD streams. If you are doing ThinClient/Thick Server computing, put 1 or more 10GbE cards in a server and connect them to a 10GbE/1GbE switch and develop your idea at a reasonable cost.
What happens to your plan if home connectors are consolidated further, as the current climate is suggesting? HDMI is now condensing up to 4K video, multichannel digital audio, copy protection, and Ethernet onto one cable and connector. The next logical (and discussed) move is to consolidate power and possibly integrated device control. You could go with what you believe will be a standard based on good evidence, but the simple fact is you don't have any influence on any body that will actually adopt the standard. HDMI is the 900 lb gorilla that has frankly bludgeoned better physical connectors and buried effective competing logical schemes.

Think of this- many datacenters are now distributing DC power either at the rack level or higher. This is done to decrease heat output in the rack, which increases efficiency and reduces heat load, all of which reduces power consumption, which arguably makes the entire data center more 'green'. What if the future home has 1 large AC-DC transformer or converter, supplying all the DC power to every electronic and many motorized components in the home. Wall warts for hand-held devices aren't needed because of contact chargers (e.g. powermat); power, data, and control can all be carried on one cable- possibly even on one wire; devices may be controlled wirelessly, or with NFC, or hardwired- what form will that take? Solid-state solar panels supply power as DC, high-efficiency wind turbines will as well, along with fuel cells, batter, TEC, ad nauseum.

Anything can change your cable standard- and I've only brought up the possibilities that are not only on the table right now, but on the plate. If you are designing a completely self-contained system- say for a starship leaving the solar system forever- go ahead and start with a blank slate and choose whatever meets your criteria. You want me to pick an entire distribution and home computing environment 20-50 years from now? Make me dick-tater of the world for life. Otherwise, no.

If you really want to be cutting edge, invest in your idea, create a prototype system, and market it. Even though what you mention is technically possible, it is currently not practical.
 
Run conduit so you can swap out later.

Fiber is not a panacea. It can be a real PITA to work with. One morning I watched a tech re-terminating a piece of fiber...ten times. Cut, polish, put on connector, test, swear, cut off $30 connector, repeat. $300 for just one line. And there are different types of fiber, some are multimode, some are not. Fiber is also very fragile and doesn't like being pulled hard or going around sharp corners. I watched some techs run a big fiber bundle in underground conduit through two 90 degree bends. My friend told them it was a bad idea and what do you know, half the fibers ended up broken.

Tech has actually been de-centralizing. Processing power is getting cheaper and more de-centralized. One reason is that centralized processing requires too much bandwidth to do things like display processing remotely. Centralized computing is so 1960s. Centralized data storage is fine, although bandwidth is still a huge issue. It takes me 6 months to backup my home server over a standard cable modem.

Building a data / processing center is pointless on the home level. If you need the processing power, contract out to Amazon or someone else who can better utilize and pay for the hardware. And massive data processing is hard - interconnect speed, synchronization, cooling, power, etc, are all why there are pros who specialize in it.

TLDR:
Don't go fiber
Run conduit for future upgrades
Run copper
Centralized processing has been on the decline for 50 years except for the biggest (and richest) jobs.
 
centralized processiong has been in decline but distrubuted computing has started making a comeback. I use 7-8 old workstations in a rack now for my sims and renderings. thats pretty much the same concept of what i want to do, i just want it to do "more"

thanks for all your inputs. nice future talk.
 
You must log in or register to reply here.
Back
Top