Fiber: Not Just for Breakfast Anymore

Wednesday, 12 November 2008


by Mike Wyant, BAVC Senior Systems Administrator

A couple weeks ago BAVC’s Executive Director, Ken Ikeda, stopped into my tiny niche in the office I share with the other members of my team.  He smiled, made a short, sardonic comment, and left with a quick request.

“Hey, I think it would be great if we could get a blog up about our fiber install.  Nothing big – and no real timeline – but it would be great to have.”

I nodded, of course, and gave my assent.

With that, and a quick smile, he stepped out of the office and went about his way once again.  I grinned to myself and went back to the task at hand – something to do with email archives, if I remember correctly – and thought little of it. That night I sat on my couch, pulled out my laptop, put my anxious fingers to the keys and… a cat sat on my keyboard, thus ending my first attempt at this article.  Since then I have attempted this blog post exactly three times, each with the same ultimate conclusion.

(I’m sure many of you would have been interested in the contents of that first article, so I have copied it at the end of this for your edification)

Today is different.

Today, the cat merely lay down on my arms, thus giving me the freedom of movement to finish this star-crossed post – and finish it I shall.

First, a very, very simplistic explanation of what fiber is and how it works.  The easiest way to start would be to point you to a holiday favorite that popped into Christmas and novelty shops in the mid-90s: the “Fiber Optic Christmas Tree.”  These trees came in many different sizes and shapes, but the most popular of these was neither Christmas-y nor Tree-y.  It was simply a bundle of what appeared to be semi-opaque strands of plastic, which were tied to some base.  This base provided rotating color schemes similar to a laid-back rave.  The thing that made these, umm, things cool was the fact that as the color changed in the base, only the very tips of the strands changed color. This is how Fiber Optic networks work as well, though it isn’t quite as pretty since you never really see it.  That and the use of lasers make it a little unsafe.

Anyway, using the same general technology as the “Fiber Optic Christmas Tree” we are able to send huge amounts of data along multi-kilometer long strands of glass using light.  It is a very cool technology and is something BAVC has committed itself – and it’s business model – to over the next few years.

You see – BAVC put forth an initiative founded within a few auspicious grants back in 2007.  This initiative would establish a fiber presence at BAVC and, using various dark technology secrets, would enable us to connect to the CENIC¹ network, which in turn would connect us to National LambdaRail².  Various folks, because of the speed and guidelines established during their creation, have referred to these networks as the “Next Generation Internet.”³  

As a speed comparison with the “Regular Internet,” standard home network/LAN (Fast Ethernet), modern home network/LAN (Gigabit Ethernet), and “Next Generation Internet,” let’s establish a basic metric:  0-10; 0 being no Internet and 10 being the fastest Internet possible at this time.  Why 0-10 instead of 1-10, you may ask?  Well… you’ll find out very quickly.

Breakdown:

10 = “Next Generation Internet” (10 Gb)

4   = Speed of high-speed SAN environment (Storage Area Network/4Gb per channel)
3   = Speed of standard SATA hard drive (Drive that most computers currently come with)

1   = Modern Home Network/LAN – In the house – (Gigabit Network/1 Gb)
.1  = Standard Home Network/LAN – In the house – (Fast Ethernet/100 Mb)
.016 = “Regular Internet” (Cable)
.006 = “Regular Internet” (DSL)

Pretty intense, huh?

If you start to scale your perception of the Internet based on this metric, it quickly gets beyond comprehension.  I mean, I’ve been working with this stuff for about thirteen months and I still can’t comprehend exactly what it means in terms of scale.  

Well, enough on that for right now.  I’m sure I will be covering that full project in more detail at a later time, but I’ve been tasked with explaining what we have accomplished thus far in our fiber implementation.

Starting in February of 2008, BAVC collaborated and ultimately contracted with City College San Francisco (CCSF) to have some of their ICONS students come out and lay down a complete internal fiber network within the BAVC office.  The process was complicated due to the number of connections we were looking for (48 pairs, or 96 actual strands), the fragility of the cabling (remember: it’s glass!), and the aggressive timeline we set (one full week of work).  We ended up going over time due to the near impossibility of pulling 96 strands of fiber in one week, but as of late April the job has been completed, due in no small part to our former Director of Technology, Chris Lincoln.

It was nearly a full week where we all just looked at our shiny new fiber and thought to ourselves, “Now this is a job well done.”  There was a lot of “King of the Hill”-type assent as we would sip our lattés or grandé-peppermint-white-mochas and discuss other geeky topics.

If you put that scene 40 years ago, replace the fiber with a rebuilt Chevy big-block, and replace the yuppie-drinks with some pull-tab Budweiser, I think you will be able to picture it quite well.

However there is always something else that needs doing; two somethings, in this case.  The first was to begin using the fiber we had just lain down.  BAVC had purchased a bunch of network equipment from Force10 Networks (one E300 router and five S50v switches to be precise), but prior to the fiber hookup, we weren’t able to really take advantage of it.  Instead, we had simply installed it and left it alone.

Our first attempt at usage resulted in a building-wide power outage.

Whoops.  Apparently our A/C system had been put on the server circuit at some point…

Following that – and an electrician visit/rewire – we brought out a Force10 engineer to walk through the programming of our network equipment.  I won’t bore you with the details here as they lack amusement and easy translation, but suffice it to say that the equipment and the programming was necessary for the whole process.

Once this was done we hooked up three of our servers via super-high-speed 10Gb cards we picked up at Small Tree Communications5 and have been humming along ever since.  We still have some network optimization to do so that we can fully take advantage of the 10Gb network, but beside that, it is up and running.

In addition to using our new fiber for ridiculously fast networking, we also used it to implement a SAN6 environment.  Essentially, a SAN is a large storage device, kind of like a really, really big external hard drive, that connects over a combination of copper networking and fiber.  By installing this fiber and investing in Apple’s Xsan7 software, we have expanded out high-capacity storage to all of our suites and made such connections available in our labs as well.

I’ll offer a bit more information as something of a teaser – and to wrap up my two part statement from above –  so that you will come back for the final installment of this two-part blog post.

The second thing took longer than expected (almost 8 months longer), but required the least amount of work on our part.  As of October 31st, BAVC has a fiber connection from our offices at 2727 Mariposa down to 200 Paul, which is where we expect to patch into the CENIC network.

Pretty exciting stuff for being so super-geeky, isn’t it?

END PART I – Fiber: Not Just for Breakfast Anymore

CAT ENTRY:
BAVC embarked on an arduous journey earlisdf cvxzcv,ma.cf[;vxckjnefjosxZ
va sdj.

FOOTNOTES:

1  www.cenic.org - Corporation for Education Network Initiatives in California: CENIC designs, implements, and operates CalREN, the California Research and Education Network, a high-bandwidth, high-capacity Internet network specially designed to meet the unique requirements of these communities, and to which the vast majority of the state's K-20 educational institutions are connected.
2  www.nlr.net - The foundation of the NLR infrastructure is a dense wave division multiplexing (DWDM)-based national optical footprint using Cisco Systems' 15808 and 15454 optical electronic systems, with a maximum capacity of 40 and 32 wavelengths per fiber pair respectively. Each wavelength can support transmission at 10 billion bits per second (10 Gbps). This optical system is deployed nationwide across roughly 15,000 route-miles of dark fiber that NLR has obtained through Level 3 Communications and WilTel Communications. Four NLR wavelengths have been implemented using 10 Gigabit Ethernet LAN PhY (physical layer), a technology and architecture that had previously been limited to metro-area networks. NLR can also support the SONET (Synchronous Optical NETwork) Technology employed in traditional telecommunications networks, if needed.
3  http://en.wikipedia.org/wiki/Next_Generation_Internet - Next Generation Internet refers to a number of projects intended to improve Internet performance and/or content quality in regions of various sizes and location.
4  http://www.ccsf.edu/Resources/ICONS
5  http://www.small-tree.com/
6  http://en.wikipedia.org/wiki/Storage_area_network
7  http://www.apple.com/xsan/

Trackback(0)
Comments (1)add comment

andyw said:

0
...
Mike,
Congrats on getting the fiber installed in the facility and the link to 200 Paul lit. I look forward to seeing the entire system functioning in the near future.

Andy
 
November 18, 2008
Votes: +0

Write comment
You must be logged in to post a comment. Please register if you do not have an account yet.

busy
 
Sign up for eNews