Avoid Growing Pains: 15 Tips to Properly Setup Your Own Hosting Racks

Server With CablesWhen you get successful, you may find that you move from someone else looking after your hosting, to running a complete rack (or racks) yourself. Standing with that shiny new rack towering over you, there seems so much space and flexibility, and anything not quite right now can be fixed later, right?

Correcting a bad installation will be extremely difficult, disruptive, or just impossible to do once you are up and running. Follow these tips in planning your rack before the first piece of equipment goes in and you will save not only time later, but you will be thanking me when the sh*t hits the fan.

1. Get Your Ventilation Right

Find out which way the servers vent, and mount them the right way round. According to the rather strict guidelines from https://path.network, if you get this wrong and not only will your servers run too hot, but you will likely be required to shut them down and turn them round – or they will be shut down for you.

Data centers have hot and cold isles. The air intake on the servers should be in the cold isle, and the exhaust should be in the hot one. All your servers should be mounted the same way round by airflow otherwise you can end up with the exhaust from one server feeding into the intake of another.

2. Choose the Right Switch

The more equipment you run, the less able you are to tell which equipment consumes which bandwidth. The best way to handle this is by using a managed switch. A managed switch will give you a facility to capture traffic statistics for each port and using a tool such as Cacti you can report on this.

3. Plan Where to Position Your Kit

Think about which equipment is likely to produce the most heat and put it near the top; not too high that its difficult to lift though. When mounting equipment on the back of the rack, think about where the servers will go and make sure you leave that space clear.

Plan networking to keep the cabling short and neat. Put the network switch at the top on the back where the servers have their networking on the rear. Then run the networking down
one side of the rack, and the power down the other.

4. Use the Proper Supports

It might be just you at the data center and if the server you need to get to is at the bottom of a stack of unsupported servers you might be in trouble.

If your servers come with rack rails, use them. Make sure your servers are properly supported so that each one can be removed independently; use rack shelves for servers without rails.

5. Plan Your Capacity

Think about the amount of equipment you will likely install in your rack, and size your network and power accordingly. By this I don’t mean bandwidth into the rack – this can easily be upgraded down the line.

What matters now is that you plan enough network ports for the equipment going in the rack. Make sure you take into account not just the servers, but other equipment such as Masterswitches, KVM etc. Its cheaper to put in a 24 port switch now, but it will be a real pain to swap it out for a 48 port one later.

6. Use Remote Power Management and Fuse It Correctly!

The ability to power-cycle your own servers either using on-board management or a remote power management unit such as the APC Masterswitch can be a godsend. For a small cost you can bypass the remote-hands of the data center and be in control of things. This is particularly important if the data center is not manned 24 hours.

If you install power distribution or management, make sure you fuse the supplies according to the load they are capable of drawing. Here in the UK, an 8-port Masterswitch is capable of handling 10 amps. Make sure its supply cable and fuse are capable of this. If your cable is fused at 5A, you won’t know about it until the fuse heats up and melts the plug; this will be a slow death and when it eventually fails, it will happen either at your busiest time, or in the middle of the night.

7. Run Dual PSU Servers off Dual Mains Supplies

If you are lucky enough to have servers with dual power supplies, make sure each supply is fed from a different power source. This could be two separate Masterswitches or even two separate incoming mains supplies to the rack. The server is designed to happily run on a single supply and so this arrangement will give you the best redundancy in the event of a failure.

To power cycle the server you will need to shut down the power to one PSU, and then cycle the other one, finally bringing the first supply back.

8. Keep Power (& All Cabling) Accessible

Once your rack fills up, the access you had to the spaces between the front and rear of the rack will be gone. Make sure the incoming mains supply and distribution will still be accessible, and your incoming cable feeds (most likely in the floor) are kept clear in case you need more cabling installed.

If your power distribution has been fitted in the middle part of the rack or behind the rear mounting rail, MOVE IT NOW – while you still can. Visualise your rack full of equipment and make sure you will still be able to access all power and network.

9. Program Network Equipment Before You Install It

Once your switch is installed and in use, you don’t want to discover it needs a reboot to configure it. Make sure switches are set up before you take them to the data center.

10. Label Everything, Front and Back

I can’t stress this too much. Invest in a Dymo and label everything on the front and the back, and if it has a cover, label that too. It might not be you visiting the equipment in the future, so it needs to be very obvious to someone who may not have seen your rack before which server you want them to look at. This will also help when you are there in an emergency and need to be sure the server you are messing with is the right one.

11. Maintain Up-to-date Documentation

Every time you make a change to your equipment, make a note of the change. If you alter power, update your Masterswitches. If you add some network, update your switch/monitoring. You can’t effectively manage your servers if you are not 100% sure you are looking at the right thing. Imagine power cycling a server, only to find you just killed the wrong one!

12. Get Your Own Monitor and Keyboard

Data centers usually provide a trolley with a keyboard, mouse and monitor – so you don’t need your own there … right? Wrong. Imagine you are there in an emergency needing access to one of your servers only to find you need a USB keyboard and the only one on the trolley is PS2. Or you turn up after a failure and there are half a dozen other engineers there fighting over the same trolley. It does happen.

Invest in your own; a number of companies make a 1U monitor with keyboard, so it won’t use up much space and its not expensive.

13. Use KVM / Server Management Over IP

It can be a very handy thing to know what’s on the console screen of the server in your rack. An IP KVM will allow you to be standing at the console of that server from the comfort of your own desk. This can be extremely useful in the diagnosis of problems, and will allow you to recover a server you just broke the network configuration on! Another way to accomplish this is with a management card installed on the server.

14. Keep Cables Tidy

Don’t just use the extremely long power cables which came with your server. Buy a range of lengths and aim to keep them as short as possible. Cable tie the power lead to the back of the server so that it can’t accidentally be dislodged. Make or shorten your own network cables so they are exactly the right length. Use plenty of cable ties to keep things neat and make sure the connectors are not under any strain.

15. Keep Tools and Spares on Site

If space allows, install a rack drawer in your rack, and keep a range of spares in it. These should include:

  • power leads
  • fuses
  • network cables
  • cable ties
  • rack nuts and screws
  • tool kit including screwdrivers, pliers and cutters
  • documentation

13 comments ↓

#1 Jeff on 09.30.08 at 4:44 pm

Another comment: do not use zip ties. They look great at first, but when you have to re-cable something, cutting them is difficult at best and damaging at worst. Use velcro-wraps instead.

#2 matt on 09.30.08 at 5:42 pm

and no matter how many velcro wraps you have, you don’t have enough

#3 Mutt on 09.30.08 at 6:08 pm

@Jeff – you can use zip ties if they have a tab that lets you release them. Much easier than velcro or cutting.

#4 Sascha Vogt on 09.30.08 at 8:16 pm

Buy the right type of rack. There are more differences between cabling racks and server racks than just the depth. A lot of my customers bought cabling racks because they found them cheaper and didn’t know that really ment.

Also leave some spare room between devices as long as you can. Unless you are filling 5 cabins with pizzas, you will be happy when you have to replace a broken box and it slides in and out easily, or the replacement is just one or two units higher than the old one.

Get and stick to the ryth’m of the holes. The gaps between them are different on purpose.

Never mount anything but switches, computers or drawers to the rails. Otherwise it will get into your way. I promise!

Color-mark your powercords. And if you need things with plugin-powersupplies, put big labels on them, which you can read in the dim light of the fully loaded cabin. Also take a note of the voltage and how much power they deliver. Buy spares for them in advance. Some of them are not easy to get, especially with the right connectors on the equipment side.

If you are going to use machines with non-standard, especially hot-plug power-supplies, buy your spares with the machine. A HP support company wanted to charge me €600,- to replace a 2,5 year old hot-plug powersupply, which HP couldn’t deliver any more.

Leave screws, a fitting driver and a flashlight in the cabin. And if your inner evil really wants to stay with zip ties, leave a cutter too.

Try to keep everything which is not 19″ mountable outside the cabin. It is the source of all (cable-)mess.

You need at least as much space in front an behind the rack as the rack is deep.

#5 Mario on 09.30.08 at 8:26 pm

Instead of zip-ties and velcro wraps, I use Twist ties. you can buy them in different colors, and in spools. Re-cabling is a snap.

#6 Adam Keck on 10.01.08 at 1:37 am

Matt:

Fortunately, thanks to diaper manufacturers, there is (finally) plenty of inexpensive effective velcro on the market. We can get 50 wraps for $5.00 or so at our local hardware store. I buy three packages at a time.

-Adam

#7 Matt Simmons on 10.01.08 at 10:28 am

Hi! Cool set of tips. I’m linking to it today on my blog. It should go live at 8:30am ET.

I also did a howto on racks and rackmounting a while back that some people might find interesting.

#8 Jonathan Clark on 10.01.08 at 4:00 pm

Thanks for all your responses.

@Jeff – I can understand not wanting to use cable ties – I have been in front of a rack trying to get a cable out with only a car key!

However, cable ties over velcro etc. not only allows you to tightly bunch cables, but it also stops them from slipping through. This is important to stop strain on the connectors.

The ability to cable tie a mains lead to the back of a server cannot be overstressed. This makes 100% sure that someone ‘rummaging’ in the back of the rack cannot dislodge the mains on a server. After you have been working in the back of a rack do you check every power lead is in tight? I bet not. I don’t have too as I know they are still secure.

See point 15 for making sure you can undo cable ties – always keep a pair of snips on site!

@Matt Simmons – thats a really great article. I hadn’t seen it before – thanks. I totally agree with the comment on cable management arms. Having previously had these in use on the back of PE1950s, PE2950s and PE6850s we found them to create hot-spots in the back of racks.

@Sascha Vogt – good points. Particularly the point about lining up with the ‘U’ of the rack. Matt Simmons post explains this well with pictures too.

#9 Dan C on 10.01.08 at 4:38 pm

Nice article. I have a couple of things to add that I hope are of use to others.

The best piece of advice I can offer is that your next install will always be better than your last. Invariably you will find yourself asking “why didn’t I do that last time?”.

That aside, use velcro ties with eyelet holes. Store plenty more in the rack than you plan to use. In addition to not being reusable and a pain to remove, nylon ties have a habit of snapping after a couple of years in situ and overzealous use can damage cables. Especially on more fussy Cat6 runs.

Where possible purchase a well constructed rack in favour of a poorly conceived pressed steel alternative. Shark make some good examples. You’ll forgive the extra expense when your cables are in neat, purpose made conduits and you can customise things like the rail and door setups with ease. Extra depth is a godsend for cable management and ventilation.

PDUs with a builtin ammeter will allow you to keep your power usage in check instead of relying on scheduled readings from your colo provider. If you’re in the UK, a company called Data&Power produce some nice units to spec.

Label all power and data cables at _both_ ends and record in a spreadsheet or network diagram. It will save you wondering which cable went where when you unplug a handful of machines. Additionally it is good for DR planning when you are offsite. Don’t change the cabling without updating the labels.

Lastly, avoid the temptation to cut all of your cables to length. It’s a brilliant idea when you first install the kit. But if you ever part-upgrade the contents of the rack then chances are the old cables may no longer reach. Try to overestimate pre-made lengths by a little bit and tie the slack someway behind the machine.

#10 links for 2008-10-02 : Bob Plankers, The Lone Sysadmin on 10.02.08 at 4:52 pm

[…] 15 Tips to Properly Setup Your Own Hosting Racks | The Matchbox Good information here, even if some of it seems pretty obvious. Do it right and keep it organized FROM THE BEGINNING. Things get really out of hand real quick, and then you spend five to ten years fighting cable tangles, etc. […]

#11 Links [2008/10/23] | Pc-aras on 10.23.08 at 5:01 pm

[…] 15 Tips to Properly Setup Your Own Hosting Racks | The Matchbox […]

#12 Hairy Airey on 11.22.08 at 11:59 pm

Be careful if you power your equipment from two sources. I once worked somewhere where they thought it was a good idea to power one PSU from the mains and another from the UPS! Be careful that both supplies are on the same phase. Worst case scenario is one faulty PSU that trips the power to the entire rack which you cannot plan for except by having two circuits to that rack. Finally, as a qualified electrician I have to tell you that you must label the equipment as being powered from more than one source.

Personally I’m curious what kind of hosting company allows a practical free-for-all of staff turning up. Most hosting companies I know won’t let you near your kit without an appointment.

#13 Abo on 11.25.08 at 9:12 pm

Try Rapstraps instead of cable ties/velcro wraps. They’re flexible so don’t pinch cables, each one can be cut and used multiple times or if you leave the tongue intact they can be re-used. Yes, they’re the Dragons Den things…

http://www.rapstrap.com/

Leave a Comment