Woe is the Network Admin?


When IT professionals huddle in our groups (gaggle? murder? flock?), one of the conversation undercurrents tends to be that we do have a bit of a rough job.  When you do your job really, really well, no one knows you’re there.  People only want to talk to you when they have a problem.  “Normal business hours” don’t mean anything – nights and weekends are just part of the territory.

It isn’t complaining – I think there’s just a sense of happiness to be talking to other people who understand and appreciate what the position entails.  To outsiders, this may sound a bit odd, or even unwarranted, given our compensation levels, but I can tell you from experience that it can be mentally challenging to constantly be in problem-solving mode.  Tack on the occasional bad manager, unrealistic budgets, crazy deadlines, and IT can be a tough place.

All that moaning aside, there’s a new study out that surveyed IT administrators regarding workplace stress.  Some key findings:

  • 68% of all IT Admins felt their job was stressful (those 32% must have it nice)
  • 49% work more than six hours a week overtime
  • IT Staff from firms of 100-250 users are most likely to quit due to stress

But the most telling statistic is this: 73% of respondents are considering quitting due to workplace stress.  That’s a huge number.  Of those, 28% cite budget reasons and lack of support resources as the primary cause of stress.

There are a bunch more statistics (41% lose sleep, 39% miss social functions, 38% miss time with their kids, 19% have health issues, etc) – but for those of you who have seen Architects on a deadline, you know those complaints are not in our domain alone.

I’ve gone on before about the difficulty in being a one-man IT shop, and the key takeaway for me from this survey was just how disproportionate the stress level is at the 100-250 sizes.  In the growth of companies, there seems to be any easy way to meet needs at the 50 person level, but as firms grow to 100-250, the recognition that the IT spend may need to grow at a level that doesn’t match the 0-50 growth may not be recognized.  You start running into IT expenses that firms of that size haven’t dealt with before.  At 500 users, the understanding of what the required spend for competent IT seems to have sunken in.

When I worked at a large Orange County VAR, the hours were crazy (24-hour shifts weren’t commonplace, but not very rare, either) and the stress levels extreme.  I would like to think – I am an optimist – that it doesn’t need to go that way.

My personal belief is that there is an existing assumption, ingrained into the industry, that we are going to work extra hours; we will occasionally have to work at night, and on weekends.  It just comes with the territory.  My goal is to say – fine, let’s say the baseline for extra, off hours is 10% of all hours, under the best of circumstances.  What can we do, from an organizational, or infrastructure perspective, to make sure that 10% doesn’t creep into 15% or 20% because of bad equipment, poor maintenance, or unrealistic deadlines?

Stressed workers don’t make good workers.  Unhappy workers tend not to stick around.  It can be difficult to assess, at times, the difference between normal-IT-stress and hey-this-guy-is-really-stressed-out-stress.  If it is the latter, the chances are high that there are steps that can be taken to lighten that load and have a more effective, balanced IT workforce.  Anything from implementing a structured maintenance plan, to rotating after-hour efforts across staff, to using the occasional outside support, to investigating and correcting time sinks, to simply forcing certain staff to learn how to say “no” when they absolutely have to.  At the end of the day, an IT group that is working hard, but healthily, will be much more effective overall for your firm.


*Insert shocking title here*


I really have to learn how to write these blog titles better.  I could learn something from Peter Hinssen, a professor at UC Irvine who recently wrote an article entitled “It Departments Have Become Completely Useless“.  That’ll grab your attention, wouldn’t it Dear Reader?

Full of rage and indignation (I’m so easy to reel in), I read the article. Through the red mist, I managed to take away a few key opinions:

First, that the title “CIO” connotates a strategic ability that most current CIOs don’t fulfill – that CIOs really just act as a technology enabler, without real peer status or impact at the C-level.  He blames this, in part, on the fact that many CIOs are in their position now because they happened to be the one guy in the office who knew about computers when computers first entered the workplace.  In the Architecture vertical, I can confirm this is fairly common.  Many CIOs were drafters or computer hobbyists who said “hey, I can get us going on computers”.

This is not to devalue their contribution or abilities – it is merely a general statement about the evolution of technology.  20 years ago, the one-eyed man was King; now, your interns have eight eyes.

That evolution – and commonality and ubiquity of technology – means that IT groups no longer have the sort of exalted, mysterious, black box magic of years past.  And while that mystery may have worked to some IT groups’ advantage, the curtain being pulled back is a good thing.  The bar is no longer being set, and measured, solely by IT.

So the “traditional” identity of the CIO, as Chief Wizard and Blackberry-getter, may not cut it in today’s expectations of leadership.  In my definition of the optimal IT group, it certainly does not. Mr. Hinssen goes on about “CDOs” that really need skills in BIg Data, digital communications and social networking (!).  His trail goes one way; mine a bit different.  I could care less if the CIO knows how to use twitter.  Do they know how the *business* works, on a fundamental level?  Are they able to marry that knowledge with their technical awareness to create solutions that impact the firm’s bottom line?  If you want to get rid of IT’s reputation as a cost center, start making money.

Mr. Hinssen’s article devolves into buzzword-blather, but at the core, he has a point.  Our industry is maturing. The percentage of the workforce with a very high comfort level with technology is constantly increasing.  The bar is no longer being set internally.  That fact can be viewed one of two ways; with apprehension and fear, or with excitement at the opportunity.  I choose the latter.

*no, his title really didn’t have anything at all with his article.  Gotta learn that trick too.

What C-Level execs *hate* to hear…


This is the first part in a multi-part string regarding “truth to power”.  I think I first heard that phrase in a West Wing episode, someone asking a potential hire about their ability to be direct with the President.  I thought to myself, hell, sure *I* can do that easy! – which is probably what got me in so much trouble in so many jobs.

See, anyone who is reckless enough or “brave” (read: naive) enough to be direct and open with leadership can do so.  But it takes another layer of nuance to be successful when speaking truth to power.  What’s the goal?  What’s the atmosphere?  What will the reaction be?  How do you craft that message in a way that it will be best received?

It’s not always easy.  Not every manager wants to hear the truth, no matter how massaged it might be before arrival.  Not always will your “truth” be their “truth”.  It is a complex process, and a delicate one.

To my headline:  in all my years, the phrase that gets CIOs, CEOs, CFOs the most riled up is “they don’t care about us”.  This phrase almost always comes up when there’s a contract negotiation going on, or when a firm is requesting something special from a vendor.

I have heard this phrase from an accountant at a 20-person law firm.  I have heard this phrase from the CIO of Isuzu.  The answer is no.  They don’t care about you.  And it drives execs insane.

“Don’t they know how much we spend with them?”  “Don’t they know we’ve been doing business for X years?” “Don’t they know how influential we are?”

It’s human nature.  In that firm, they *are* important.  Movers.  Shakers.  Etc.  They’re used to getting their way.  But that circle of power has a very clearly defined edge.  It was always weird for me, as a VAR, to walk into a big firm, and see the aura of deference given to the top executives, and being immune to it.  To me, they were just another guy at just another firm, but to their staff, they held the ability to hire and fire.

So if you’ve gone to a vendor, asked for something, and they said no, the answer is, they don’t care.  Vendors don’t like saying no.  It doesn’t come easy.  Whoever your firm’s contact person is, they ran it up the ladder at least one rung making sure it was OK to say no.  So the vendor has weighed the options, and decided no.

There are ways to determine your importance.  This can be directly measured by the amount of resources a vendor assigns to you.  If you have a *dedicated* account manager – or better yet, a dedicated account team – you’re important.  If HP has a technician with a one-to-one mirror of your CEO’s home server setup for help with troubleshooting, you’re important.

So you’re saying, great, we know we’re not important, now what?  The big hurdle is acknowledging that and moving past it – and figuring out how to game the system.

First, maximize your purchase power.  I’m a big believer in consolidating spending when possible.  You may still be tiny in the big picture, but you’re important to a sales rep somewhere.

Second, if you like to have lots of influence over vendors, pick very small ones.  I speak from experience when I say that small vendors will jump through hoops that large ones wouldn’t even blink at.

Third – leverage *their* needs.  This comes in two forms.  The easiest is end-of-quarter and end-of-year quotas.  You suddenly become a lot more visible on their radar when they have a number they absolutely, positively have to hit – and you’re the one who can get them there.

This goes for non-sales metrics for your rep as well – i.e, onsite visits, lunch and learns, etc – find out what they’re measured on, and how you can help them with their internal numbers.

Fourth – try to form personal relationships.  Have lunch meetings.  Get to know your rep.  With a personal connection, they’ll fight just that much harder for your request.

Using one or more of these methods can help you increase your firm’s leverage, and help your exec (or you) get what they want.

Postmortems: they ain’t just for dead folks


If you’ve seen a cop drama in the last twenty years, odds are high you’ve watched a scene where the detectives and the coroner stand over a body on a stainless steel table.  Give us the straight scoop, doc, how’d he catch it?, they say (at least if the show is in black and white, anyway).  You think I’m a miracle worker? retorts the coroner.  Come back tomorrow and I might – might be able to fill you in.

The TV never shows what the coroner actually *does* – rarely anyway – because A) it’s really gross, and B) more entertaining to watch the detectives chase someone down.  But in the real world, the job the coroner does is critical to any crime investigation. How was the person killed? How tall was the killer?  What hand did they use?  What was the weapon?  Etc, etc.

Now, failure in the IT world can be pretty ugly.  Email went down.  Phones went down.  Files went missing.  Revisiting those wounds isn’t always an easy thing to do.  But if your IT group is healthy, and clicking on all cylinders, project and incident postmortems should be a regular part of your routine.  There are, however, a few rules to follow.

Number One: let some time pass.  This is tricky, because you have to walk a fine line between allowing enough time to pass to allow emotions to settle down, to let all parties (IT and non-IT) to return to normalcy, but also close enough to the event where everyone’s memory is clear about what happened.  If you’re going to err, err on the side of waiting longer for cooler heads to prevail.  By nature, a postmortem of a negative event has the potential to become emotional.  Keep the personalities involved calm.  This isn’t an opportunity to beat up on the guy who accidentally pulled the plug out of the wall.   Use the postmortem as a learning event.

Number Two: have a set procedure, and follow it.  What exactly happened?  What could we have done to prevent it?  What process needs adjustment, if any?  Do the costs of implementing preventative steps outweigh the costs of the event happening again?  Maintaining a format allows people to know what to expect, and maintains a sense of impartiality.

Number Three:  maintain accountability.  Not always possible, but don’t use a group setting of a postmortem to hide the fact that there’s a specific problem, or error, that occurred.  This is especially critical when a postmortem includes another business unit.  If it is the IT Group’s fault, take responsibility.  Sugarcoating, or passing the buck, won’t help you get to that Trusted Partner status.  This must be managed; postmortems are not a place for people to beat up on someone.

Number Four: follow up.  If the postmortem conclusion has a specific action item (i.e., move patching from weekly to monthly), do it.  If the p-m process becomes a show-and-tell display rather than a meaningful process, people will quickly learn to not approach it seriously.

Number Five – and perhaps the most important: do postmortems after every project, even successful ones.  For successful p-ms, the tone and approach are different; more of a “what could we have done better” attitude.  These are also a chance to give credit where credit is due.

If you think about it, the postmortem is really just an exercise that increases communication levels, be they internal to IT or with a business partner.   Of course, it is difficult to conduct these when you’re running around putting out fires or running from one project into the other.  As with all processes that are higher up Donahue’s Hierarchical Pyramid of IT needs, they rely on the fact that the lower levels are being adequately met.  But postmortems are a key indicator that an IT group is in a good place.

Gratuitous Thanksgiving Day Post


Ok.  It looks at the calendar.  It smells the turkey topping on the Domino’s pizza.  It pulls out the Underwood typewriter.  It writes the Gratuitous Thanksgiving Day Post.

So, once a year, I have to be thankful?  I can manage that.  Do I have to *pay* anyone?  No?  Perfect.  Are my thanks an indicator of future success or benevolence? No?  Perfect.  Will people read this and think how humble and great a person I am? Yes? Perfect.  Let’s get this over with, then.

I should thank WordPress for giving me the platform to write this with, but then I’d get into this loop/chain/black hole of thanks that includes AT&T for the connection, SoCal Edison for the power, my mother and father for not cutting off my fingers as a child, the United States for my unalienable rights, and the Creator for what passes as a marginal IQ, so I won’t thank them at all.

I *am* thankful for Google.  Sweet, clean google.  Savior of a thousand technical problems, high and low.  I’ve said a hundred times (mostly to the same people, over and over again ad naseum) that the key to being a good tech is being a good Googler.  My “thanking chain” logic says I should really be thanking the folks that screwed something up before I did, and took the effort to post about it, and even moreso the people who posted the fix, but that’s too much effort.  I will thank whoever the hell Boolean is, though.

I *am* thankful for the Internet.  O Deliverer of Porn, Solver of Problems, Bringer of Reddit.  Our calendar in a thousand years should reference B.I. and A.I.  I shop for toasters by including the term “wifi”.  I rage if the TV I want doesn’t have Netflix embedded, even though my blu-ray and my a/v receiver already do.  I’m not thanking Al Gore, I’m not thanking Darpa.  I will thank the “bunch of tubes” guy because he made it easy to explain the Internet to my parents.

I *am* thankful for Microsoft.  Yes, they gave us BOB, and WinME, and Windows 8, but they also gave us Windows 95, Windows XP, Windows 7, Server 2008, Exchange, Outlook, and Minesweeper.  Yes, it’s easy to kvetch about the Evil Empire, but I’m glad we have, for all intents and purposes, a single vendor for desktop OSes.

I *am* thankful for Mac Fanatics.  They make me feel grounded.

I *am* thankful for tech support in the United States.  My English is worse than my Canadian, French people think I’m speaking Spanish and Spanish speakers think I’m speaking German.  So I’m behind the 8-ball (il ocho oeuf) to begin with, and I don’t need Sachin Tendulkar’s second cousin telling me that his name is really Fred.

I *am* thankful for the career mistakes I’ve made, at least the ones that didn’t get me fired.  Mistakes – be they technical or managerial or operational – still provide the best learning experiences.  And boy, have I learned a lot.

That’s it.  I can’t think of a single other thing I’m thankful for.  How about you?

Sometimes, you just need a win


For better or for worse, success in IT is perversely  measured; the height of success is the height of ubiquity.  When Lord Grantham reaches in his coat pocket for his smoking-pipe, he doesn’t even think about it; but Bates anticipated the need earlier in the day and placed it there.  Sorry, I’ve been watching too much Downton Abbey for my own good.

IT is like that ubiquitous butler.  Things run smoothly *not just because*, but because of time and effort put in by silent stewards.  And in IT, our presence is most often requested when something is amiss.  We can work fiendishly to make sure that email server has 99.9999 percent uptime, but when it goes down on a Tuesday at 2 pm, watch out!

This is not a complaint, or a whinge, as the Earl would say.  It’s just how things are, and those in IT accept it as the nature of things.  People come to us when things don’t work; we are, by circumstance, associated with problems, and problem-solving.

All that said – that constant refrain of “when I’m doing my job really well, no one notices, and when something goes wrong, I’m the first they call” – while accepted – can be a negative over time if it is the *only* form of relationship between a firm and its IT staff.  Ubiquitous service is its own reward, but even the Butler gets to suit up for the cricket team and have a day in the spotlight.

Stop with the freaking analogies, you’re thinking, and speak plainly, man!  Sure.  Every once in a while, the firm needs to either introduce new technology, or upgrade a process, during which IT gets a “win” under it’s belt.  These actions – outside the realm of day-to-day maintenance – reinforce the perception that IT is good for the firm, and allows IT to continue along the path to Trusted Partner.

While a recent example of this for a client was a whole new phone system, these projects don’t need to be expensive.  A new open-source IM solution, or a firm-wide open source Wiki.  Usually the best ideas for Wins inside an organization come from the users or business unit heads.  It’s all about communication and creatively solving problems – the same as our ubiquitous butler, but on a macro scale rather than micro.  And it lets the users know – hey, you’re not just the monkey who swaps out my toner cartridge.

Fighting for these kinds of projects will keep IT morale up, will keep the staff engaged, and will help improve firm-IT relations.  They’re a key ingredient to having a happy, healthy IT group.

IT 2.0


In many ways, we in the IT industry are pretty lucky to be right here, right now.  We’re present at the birth of an industry that will forever change the world (and though I’m prone to hyperbole, that’s an understatement).  Two hundred, three hundred years from now, history will look back at the 90’s/oughts as when everything *changed*.

We’re kind of like the first employees of Edison Electric, or Bell Telegraph and Telephone, Wright Brother Airlines.  We’re able to witness, first-hand, the birth and first tentative steps of an industry.  And while we may * perceive* tech to have been around forever, to have made great and purposeful strides, the reality is that 25 years is nothing. In the great scheme of things, nothing at all.

We’re only now emerging from the embryonic stage, and the trademark for that is the passing of the 1.0 pioneers.  Gates has retired. Jobs is dead.  Ellison is active, but nearing 70.  Phillipe Kahn now dabbles.  Steven Sinofsky just left Microsoft.  These are huge, huge names – the Railroad Barons of our century – and they’re exiting the stage.  It’s as good a time as any to mark the end of 1.0 and the start of 2.0.

So what lies ahead?  Perhaps more importantly, *who* lies ahead?  Will the rules – as ethereal as they may be – that governed IT 1.0 still apply to 2.0?  Will the cyclical nature (client-server, server-client, back to client-server) continue? Will it still be a rule that marketplace dominance (WordPefect, AOL, MySpace) mean very little?  What’s the long term impact of everything and everyone being constantly connected?  What are the cultural and infrastructure implications of emerging tech – self-driving cars as a single example?

More importantly – we have our first generation that grew up with the internet starting to hit the streets.  You’ve heard the generalizations – email is too slow, everything txt, 2.0 rules, 1.0 is horrible, why can’t i schedule my toaster online?

You know, from experience, that there’s a general correlation that the easier something is for the user, the more effort needs to happen behind the scenes.  It’s a ratio that’s consistent in infrastructure design, application design, user interfaces, etc., etc.  And with the user *expectation* quickly becoming, “it needs to be easy, regardless of what I’m trying to do” – it means that more and more effort – more thought – need to go into providing that sense of ease.

There’s an unspoken reality that goes with the saturation of IT into people’s personal/home lives, and the incredible effort consumer product manufacturers put into things being “easy”.  With IT 1.0, people were just happy to have the tech, and would put up with large headaches to get that functionality.  Not with IT 2.0.   IT is no longer the domain of the business world, and the bar is being raised – and not by us.  Like the industry itself, it’s time to change.  To step up our game.  To start casting aside old mentalities, old rules.  Time to shift.