Remote Hands

I used to be a telecommunications technician.   I had numerous responsibilities, such as, helping to build out massive networks.  They called them, "MANs," short for Metropolitan Area Networks.   I liked that term, because it was a decent representation of our company's footprint.

Our company, Yipes Telecommunications, started with this idea to replace existing SONET and ATM protocols, with this thing called Gigabit Ethernet.    They secured about a hundred million dollars in angel capital during the dot-com boom, around 1998.   They used the money to lease strands of existing fiber optic infrastructure in dozens of cities in the U.S.

In San Diego, for example, they leased 24 strands of fiber optic capacity, and the glass formed a gigantic ring in and around the whole city.  Then, Yipes would have a contractor intercept the fiber.   This meant pulling last-mile fiber from the utility location, into a building.

We would call this, "lighting up," a building.

That's a very expensive process, and requires expertise in Outside Plant, ie, construction, permitting, and working closely with existing utilities, such as legacy telco plant.

You have to first find a pathway from the closest manhole, into the building, and then you run a segment of fiber from the manhole into the building and terminate it in a secure cabinet.   The cabinet contained the hardware, like a Black Diamond or Juniper switch or router.   Then you just use a patch cord from the patch panel to the equipment.

After this preparation, the contractor is tasked with intercepting the metro fiber, usually down in a man-hole, and always in the middle of the night.

I was generally responsible for helping to choreograph this upgrade, and to communicate progress and any issues to our Network Operations Center.   We just called it the NOC, and it was staffed 24/7 by network engineers.   If there was an issue that a network engineer needed assistance with, it was quickly escalated up the chain of command until the issue was resolved.

I was a technician, but to the network engineers, I was known as, "remote hands."   This is because I would have to roll out with special equipment whenever there was an unexpected issue.

I had testing equipment of all sorts.   I could check continuity of a strand of fiber up to 70 kilometers, using an OTDR.   This device had an LED screen, and it showed you a visual representation of an entire strand of fiber.   It showed you every little nuance in the fiber.   I could see the reflectivity drop off at kilometer 30, and I knew that was where the fiber had been spliced.

I would also roll out with a tester for just about any communications protocol on our network.   I spent a lot of time doing head-to-head testing with some other technician for PacBell or whatever, and together, we could determine exactly whose piece of equipment was malfunctioning.

I got calls at all hours.   I had to stop drinking, because I was the only technician serving Los Angeles and San Diego, for like six years.   But it was great, because there were many times when I became the hero.   One time we had a driving rainstorm, and I was dispatched to check on our MPOP in Riverside, because it was located inside a power substation.   When I arrived, our MPOP was high and dry, but, to me, it still felt like hero time.

A lot of my routine was just turning up a new customer, usually using a last-mile solution that didn't involve fiber, but a traditional copper router that we VPN'd into our network.   A lot of times, it's just too expensive to run fiber to a new office building.

I had a lot of respect for the network engineers.   I remember one guy who used to be a maitre'd at a high end Bay Area restaurant.   He just wanted to be a network engineer, and he was damn good.   These guys depended on me to get on-site quickly in case of an outage.   Hero time.

I was dispatched from San Diego to Los Angeles, in the middle of the night, several times.   It got to be so routine, that I had dreams of servicing telecommunications equipment in deep space.

That's no joke, and they weren't nightmares, they were awesome dreams.   Imagine being strapped to a rocket and dropped off hundreds of millions of miles away, to work on some satellite transceiver.   That was me.   Remote hands.

And all of this was just to service a customer who could not afford even one millisecond of down-time.

So, this has made me think about something.   I'm not a telecom technician anymore, but I am still a remote hands tech.   I sort of answer to a much higher network operations center now, the Highest.   And the big guy depends on me, just like the Yipes NOC once used to.

The end-user is out there.   The person who needs to know that his connection to our Source is going to stay secure, and I'm going to go wherever, whenever, and make sure that the signal strength is ok, and that the end-user's gear is receiving properly.

Hero Time.