Verizon 140 West Street, New York

Impact to Verizon Network at

140 West Street, New York, NY following Terrorist Attack

 

                                

 

 

 

 

 

 

 

 

 

 

 

Report No. 4343/01-209

January 17, 2002

Prepared by:

Telcordia Technologies

 

 

Telcordia Technologies, Inc. and Verizon Confidential — Restricted Access

This document and the confidential information it contains shall be distributed, routed or made

available solely to authorized persons having a need to know within Telcordia and Verizon,

except with written permission of Telcordia.

 

An SAIC Company

 

1.0     – Introduction

2.0     – Graphical Summary of Recommendations  

2.1             – Discussion

2.2             – FLOOR 1 – Main Distribution Frame

2.3             – FLOOR 2 – Toll and Transport Equipment

2.4             – FLOOR 3 – Power Room

2.5             – FLOOR 4 – Toll and Transport Equipment

2.6             – FLOOR 6 – Power Rooms

2.7             – FLOOR 7 – 5ESS and DMS-100 Switches and Collocation Space

2.8             – FLOOR 8 – Distribution Frame

2.9             – FLOOR 9 – DMS-100, Power Room, Cosmic Frame and Collocation

    Spaces

3.0      – Floor 2 – Assessment of Toll and Transport Equipment

4.0      – Floor 4 – Assessment of Transport Equipment and Toll Distribution Frame

5.0      – Floor 7 – Assessment of 5ESS and DMS-100 Switch Rooms and Ancillary

         Spaces

6.0      – Floor 9 – Assessment of DMS-100 Switch Room, Power Room, Cosmic

         Frame, Voice Mail and Assorted Collocation and Storage Space

7.0      – Floors 1, 3, 6 and 8 – Assessment of Distribution Frames & Power Rooms

8.0      – Chemical Analyses

9.0      – Vibration Concerns

10.0             – Thermal Concerns

11.0             – Cleaning Procedure for Asbestos Containing Particulate

11.1       – 2^nd Floor D4 Channel Bank

11.2       – 7^th Floor Lucent 5ESS

11.3       – 7^th & 9^th Floor Nortel DMS

11.4       – Distribution Frame

11.5       – Power Rooms on Floor 3, 6 & 9

12.0             – Conclusion

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 1.0

Introduction

 

 

 

 

 

 

 

 

 

 

 

 

On September 11, 2001, the Twin Towers of the World Trade Center (WTC) experienced terrorist attacks resulting in severe fires that led to their respective collapses. The structural failures caused significant collateral damages to surrounding facilities resulting in fire, and partial and total building collapse. In addition, various degrees of structural damage, water damage, and contamination occurred to nearby buildings, that included the Verizon Communications Inc. office located at 140 West Street, New York City, NY.

 

The 140 West Street facility occupies a city block that is fronted by West Street, with Barclay Street to the north, Washington Street to the east, and Vesey Street to the south. The Vesey Street side of the building faces the partially collapsed WTC 6 building and the collapsed Twin Towers (WTC 1 and 2).  Debris and steel girders from the collapsing buildings broke numerous windows and penetrated the Vesey Street side of the building in several locations that resulted in ingress of debris and contaminants. The Washington Street side of the Verizon building faced WTC 7 that collapsed shortly after 5:20 PM on September 11.  Debris from WTC 7 fell onto the side of the Verizon building causing significant physical damage that included breaches to the east wall, areas of floor slab collapse, facade damage, and numerous broken windows.  Concurrent with the building impact, structural damage, and vibration, the telecommunication network and support equipment experienced physical damage, loss of environmental control, exposure to untempered outdoor air, severe particulate contamination and smoke ingress. In addition, water damage to equipment and facility occurred from firefighting activities conducted outside and within the Verizon building. Photographs 1-1 through 1-4 illustrate the proximity of the debris and some of the external damages to the facility. Figure 1-1, reprinted from cnn.com, shows the location of the Verizon building in relation to the WTC and World Financial Center (WFC) Complexes.

 

                                             

Photo 1-1.  Vesey Street (south side)       view of the Verizon building. The red arrow denotes destroyed WTC       Building 7 on east side. The yellow arrow shows ironwork that damaged       (penetrated) the building in several locations.

           

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                         

Photo 1-2.  West Street side of the      building open to 1st floor MDF. Partially collapsed WTC 6 is in      the background.

    

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                                

Photo 1-3.   East side sustained the       most damage from WTC-7. Arrows denote continued fire fighting activity on       9/14.

                                                    

Photo 1-4.  Numerous windows were       blown in on the south side along with several wall breaches.

       

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                       

Figure 1-1. Red arrow shows the Verizon 140 West Street location in reference      to the World Trade Center and World Financial Center Complexes.  Figure 1-1 is reprinted from and      available at http://www.cnn.com/SPECIALS/2001/ trade.center/damage.map.html

      

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In response to the disaster, representatives from Verizon and Telcordia Technologies evaluated the impact to the Central Office portions of the facility located on floors 1, 2, 3, 4, 6, 7, 8, and 9 of the 32-story building.  These areas housed cable distribution frames, power, toll, transport, switching, and network support equipment.

 

The preliminary evaluation was performed September 12 through 14, 2001 and our initial report with recommendations for temporary restoration and recovery was presented in Telcordia letter 4337/01-198, issued on September 16, 2001. Our initial chemical analyses report was presented in Telcordia letter 4337/01-200, issued on September 19, 2001.  These reports supplemented information that was verbally provided to Verizon to assist in determining the feasibility of and the physical requirements necessary for the short-term operation of compromised equipment. This effort focused on assisting Verizon in the rapid restoration of as many network services as possible.

 

Since our initial activities, Telcordia has performed more extensive physical and chemical evaluations of equipment to help Verizon determine the long-term prospects for total network service restoration.

 

All Telcordia on-site assessments were comprised of visual evaluations of network, power, and support equipment for physical and/or water damages with photographic documentation. In addition, multiple samples were collected for supportive qualitative and quantitative chemical analyses.  Key issues that were evaluated for short- and long-term equipment operation include:

 

• Quantity and distribution of particulate contaminant
• Corrosive potential of particulate contaminant
• Asbestos fibers
• Quantity, distribution and corrosive potential of gas phase products of combustion
• Water and elevated relative humidity exposure including electrolytic and chemical corrosion processes
• Electrical and thermal stress
• Physical damage to equipment, hardware and surrounding facility
• Minimization of cross-contamination
• Equipment cleaning feasibility
• Restoration to a pre-event condition

 

In addition, this report raises concerns for continued network operation and recovery based on the following:

 

• Replacement requirements of floor slab sections on floors 4, 7 and 8
• Reported replacement requirements of Washington Street side building columns 26 and 40 from the base plate to the 11^th floor and columns 53 and 60 from the base plate to the 6^th floor
• Equipment relocation requirements during structural repair
• Craft inaccessibility of equipment for maintenance, diagnostics and service upgrades during structural repairs
• Vibration concerns and monitoring during structural repair
• Equipment interoperability/cabling requirements for relocated and unusable equipment
• Operation of network equipment outside of its intended design criteria as compared to Telcordia Generic Requirements and Industry Standards
• Cumulative effects of the above

 

All physical and chemical analyses data were coupled with the various concern issues to formulate the Telcordia Technologies recommendations on the disposition of equipment and the practicality for cost-effective equipment recovery versus replacement.

 

Section 2.0 graphically represents the Telcordia “floor by floor” remediation recommendations.  This section provides a summary overview via color-coded schematics with accompanying legend to assist Verizon and its Insurers when evaluating considerations for network restoration.

 

Sections 3.0 through 6.0 provide our detailed assessments of the most seriously impacted network and facility elements on floors 2, 4, 7, and 9.  Specific recommendations are provided based on the details of the extent of damages and supported by representative photographs and chemical analyses.

 

Section 7.0 provides details for recovering power rooms, distribution frames, and facility located on floors 1, 3, 6, and 8.

 

Section 8.0 provides our complete chemical analyses data including qualitative and quantitative testing to characterize the types of, and concerns with the contaminants and corrosion products. Over 250 samples were collected for analyses. Tests included Ion Chromatography, Environmental Scanning Electron Microscopy with Energy Dispersive Spectrometry, insulation resistance versus relative humidity (hygroscopicity of contaminants), pH, and copper mirror corrosion. Details are also provided on quantitative PLM asbestos analysis per New York State ELAP 198.1 method performed by EMSL, an independent laboratory, and non-quantitative assessment at Telcordia via Electron Microscopy for determining the presence of asbestos fibers in dust.  The distribution of asbestos throughout network equipment areas of 140 West Street is documented in Section 2.0 and in microphotographs.

 

Section 9.0 addresses our concerns with the anticipated vibration impacts to equipment during facility restoration.  This section offers guidelines for monitoring vibration and applies realistic limits and trigger levels for reliable equipment operation during anticipated vibration events.

 

Section 10.0 addresses thermal concerns associated with the significant frame filter loading, thermal blanketing of electronic components, and loss of environmental control. The anticipated internal frame temperature above ambient is presented based on the manufacturers reported frame power dissipation in watts per square foot (w/ft²). Test frames were measured in the Telcordia Red Bank, NJ switch laboratory with simulated frame filter loading to provide baseline data.

 

Section 11.0 provides detailed equipment and facility cleaning procedures including vacuum cleaner requirements and ESD protection for working on sensitive electronic equipment. Procedures are included for 5ESS, DMS, D4, cable distribution frames, and power rooms.  The detailed procedures can be modified following review with Verizon to include any equipment located within 140 West Street. These procedures were developed specifically to include asbestos containing dust and are based on test cleaning performed on-site or in our laboratories.

 

Section 12.0 Conclusion.

 

The reliability and survivability of the Verizon network and infrastructure is credited to the detailed engineering, documentation, maintenance, network redundancy, dedicated personnel, and the use of equipment that meets or exceeds the stringent Verizon, Telcordia Technologies, and “best in class” industry and vendor standards.  To this end, all parties in review of this document must be cognizant of the challenges to maintain network services while facility repairs and replacement equipment are engineered and procured to provide a seamless transition from compromised network elements to a highly reliable network as outlined in Verizon’s charter.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 2.0

Graphical summary of Recommendations

 

 

 

 

 

 

 

 

 

This section presents schematics of the Telcordia “floor by floor” replacement and remediation recommendations for compromised equipment located on floors 1, 2, 3, 4, 6, 7, 8, and 9 at 140 West Street.  The following schematics are color coded to aid in understanding our recommendations. The color codes are as follows:

 

Red – Denotes equipment that has been severely contaminated by asbestos containing particulate and/or physically damaged and/or water damaged and/or exposed to unusual environmental/thermal conditions. Also, included in the red color code is equipment that will require removal to permit structural repairs of exterior walls, columns and slabs.  We recommend asbestos level compartmentation with strategic placement of 0.3 micron negative air machines, or detailed HEPA cleaning of these areas until equipment is removed to minimize cross contamination and/or personnel exposure.

 

Orange – Denotes equipment that has been contaminated by asbestos containing particulate and/or exposed to unusual environmental/thermal conditions. Technically this equipment can be internally and externally cleaned to remove asbestos containing dust. Detailed cleaning would require various levels of equipment disassembly to access internal and external surfaces as well as cleaning of power and communication cables and connectors and surrounding facility. Portions of this equipment may require removal to facilitate structural repairs, e.g., 4^th floor Alcatel and Titan equipment in RR 4300 line-ups and/or require replacement since craft will not have access during the structural repair phase. The inaccessibility by craft will leave the equipment unmanageable for reliable network operation. Also, methodologies for much of the equipment located in this color code on floors 4 and 7 must be developed to minimize structural repair induced vibrations from negatively impacting equipment.

 

Yellow – Denotes equipment that requires cleaning of all accessible external surfaces including power and communication cables and connectors and surrounding facility.

 

Additional issues of concern include: maintenance, reliability, service assurance and cabling of equipment located between the different color coded zones. In particular, the costs associated with power and communication cable cleaning, cable management and mining (removal of inoperable cables) to avoid cable rack overloading, and cable rerouting must be assessed. Furthermore, all costs for detailed equipment cleaning must be established to finalize decisions related to equipment restoration versus replacement in orange and yellow coded zones. Cost consideration must also be given for the development of Methods of Procedures (MOPs), network services interruptions during cleaning, limited access “maintenance window” for cleaning, and Verizon personnel resources to oversee equipment cleaning operations and logistics, e.g., placement of equipment in and out of service as well as verification of post cleaning functionality.

 

It is imperative that Verizon Network, Real Estate, Environmental Health and Safety (EH&S) organizations and contractors coordinate all cleaning and equipment removal/disposal activities and pre-determine the post cleaning test criteria to ensure the appropriate level of asbestos abatement. It is equally important that all abatement operations be conducted with appropriate cleaning equipment and in a manner consistent with sensitive network equipment as outlined by Telcordia. Refer to Section 11.0. Failure to adhere to these requirements will void warranties and result in hard and soft equipment failures, erratic or intermittent network service, diagnostic difficulties, and likely cause a misinterpretation as to the ability of the compromised equipment

to provide reliable short-term service.  

 

 

           

Legend           Red:     Replace due to physical & water damages and contamination.     Inaccessible during building structural repair.           Orange: Technically can be HEPA cleaned.  However, it is currently unknown if the     frame will remain accessible during structural repairs.  If frame can remain, HEPA clean all frame     room facility surfaces, frame cables and terminations to remove asbestos     containing particulate.

 

 

 

 

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     

Legend           Red:     Replace due to water &     physical     damages, severe asbestos     particulate     contamination, and     environmental     excursions. Also,     inaccessible     for maintenance and     diagnostics     during building     structural     repair and vibration during repair is     expected to exceed NEBS design criteria. Recommend     detailed cleaning of facility surfaces, cabling and external and internal     equipment including circuit packs, or     provide asbestos level compartmentation     until equipment is removed to minimize cross     contamination and personnel exposure.           Orange: HEPA clean all facility surfaces, cabling and external and     internal equipment surfaces including circuit packs. This detailed cleaning     requires partial disassembly of equipment, and must meet all Verizon     Network & EH&S requirements. Equipment replacement may be     warranted based on Verizon costs for restoration, including all     interoperability & re-cabling issues as well as vibration concerns     during facility repair.           Yellow: HEPA clean all external equipment, cabling and     facility surfaces.

                                                                                                                               

 

 

           

Legend                 Yellow: HEPA clean all external power room equipment and     facility surfaces and tacky cloth or damp cloth wipe batteries and stands     to remove trace asbestos containing dust.     These     operations must be performed by persons knowledgeable with working around     power equipment, including all ESD precautions for batteries.

 

           

Legend           Red:     Replace due to water &     physical     damages, asbestos     particulate  contamination, and     environmental     excursions. Also,     inaccessible     for maintenance and     diagnostics     during building     structural     repair.     Recommend asbestos level compartmentation until equipment is     removed     to minimize cross contamination and personnel exposure.           Orange: HEPA clean all facility surfaces, cabling and external and     internal equipment surfaces including circuit packs. This detailed HEPA     vacuum cleaning requires partial disassembly of equipment, and must meet     all Verizon Network & EH&S requirements. Select equipment     replacement, e.g., RR 4300 line-ups, may be warranted based on facility     repair requirements and/or costs for restoration, including all     interoperability & re-cabling issues as well as vibration concerns     during facility repair.           Yellow: HEPA clean all external equipment, cabling and     facility surfaces.

                                                                                                                                                           

 

 

           

Legend                 Yellow: HEPA clean all external power room equipment and     facility surfaces and tacky cloth or damp cloth wipe batteries and stands     to remove trace asbestos containing dust.     These     operations must be performed by persons knowledgeable with working around     power equipment, including all ESD precautions for batteries.

 

           

Legend           Red:     Replace DS2 5ESS     due     to direct physical damages,     chemical     corrosion, asbestos     particulate     contamination,     environmental     excursions     including     elevated frame     temperature,     thermal shock and     vibration     that have likely     compromised     service reliability. Also,     inaccessible     for maintenance and     diagnostics     during building structural     repair.     Consider asbestos level     compartmentation     or detailed HEPA     cleaning     until equipment is removed to     reduce     cross contamination and personnel     exposure.           Orange: Technically can be HEPA cleaned.     However,     DS1 5ESS replacement may be     warranted     based on     costs for restoration     including     all maintenance, interoperability,     cabling     issues, inaccessibility and vibration     concerns     during facility repair. If cleaned, do all     facility     surfaces, cabling and external and internal     5ESS     surfaces including circuit packs. This detailed procedure requires partial     5ESS disassembly, and must meet all Verizon Network & EH&S     requirements.           Yellow: HEPA clean all external equipment, cabling and     facility surfaces.

 

           

Legend                 Yellow: HEPA clean all external power room equipment and     facility surfaces and tacky cloth or damp cloth wipe batteries and stands     to remove trace asbestos containing dust.     These     operations must be performed by persons knowledgeable with working around     power equipment, including all ESD precautions for batteries.

 

 

 

           

Legend           Red:     Replace DMS due to asbestos particulate contamination, and     environmental excursions including elevated frame temperature, thermal     shock     and vibration that have likely compromised service reliability. DMS will     likely be inaccessible for maintenance and diagnostics during building     structural repair resulting in prolonged service outages. Also, we anticipate     vibration/shock generated during building repair will exceed the NEBS     design criteria. Consider asbestos level compartmentation of switch space     or detailed HEPA cleaning until equipment is removed to reduce cross     contamination and personnel exposure.                 Yellow: HEPA clean all external equipment, cabling and     facility surfaces.

                                                                                                                                         

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 3.0

FLOOR 2

Assessment of Toll and Transport Equipment

 

 

 

 

 

 

 

 

Overview: Floor 2 Toll Equipment Room

Damages to the Washington Street (east side) and Vesey Street (south side) of the Verizon building resulted in breaches into the toll equipment room allowing ingress of large amounts of particulate and debris along with water, water mist, and water vapor from firefighting activities.  Our analyses of the particulate, described in detail in Section 8.0 of this report, determined that these residues were comprised of oxides of silicon, calcium, aluminum, iron, magnesium and calcium sulfate, with low levels of chlorides and potassium. That is, the particulate that has severely contaminated network equipment is primarily pulverized concrete, wallboard, and plaster co-mingled with various fibers.   In addition, quantitative and qualitative analysis of the particulate has shown that varying levels of asbestos are contained within the particulate. Other concerns raised by the chemical analyses, applicable to all contaminated equipment, include the potential for chemical corrosion of metals in the presence of moisture due to elevated alkalinity of the dust, as evidenced by irreparable corrosion product on metals exposed to water.

 

Equipment line-ups closest to the east wall included D4 channel banks, SMAS relays, MFT bays, Lucent SLC Series 5 bays, and Toll Main Distribution Frame (TMDF). There is a discernible gradient of particulate with frames closest to the wall, in line-ups RR2002 through RR2013, displaying the heaviest contamination. In several locations in this area, the particulate contamination is strongly adhered to the electronics and TMDF. This is attributed to water mist and conditions of high relative humidity during periods of firefighting and loss of environmental control after the September 11 attack. The level of contamination is reduced with distance from the Washington Street wall, but a layer of dust and asbestos is still evident on all of the equipment. Additionally, select equipment frames in the area sustained varying amounts of water damage resulting in chemical and electrolytic corrosion of active electronics as well as rusting of equipment frame and shelf hardware. Shorts and shunts associated with the water damage likely resulted in additional damage to electronic components in affected equipment as well as interconnected electronics.  

 

Summary of Conclusions

• D4 Channel Banks:  Replace all D4 channel banks in RR2005      through RR2013, and RR2100 through RR2111. Replacements are required due      to severe particulate entrainment within equipment, strongly adhered      particulate on electronics that is extremely difficult to clean, corrosion      resulting from contact with water, and physical damages. This includes all      D4 frames within approximately 80 feet from the Washington Street wall.
• SLC Series 5 and Telecom Solutions Clock      Frames:  Replace SLC 5 frames in      line-up RR2112 and the Telecom Solutions clock in line-up RR2114 due to      widespread electrolytic and chemical corrosion damage resulting from      direct contact with water.
• MFT Bays: Replace all MFT bays in line-ups      RR2002-RR2005 due to water damage and strongly adhered, “caked on”      particulate that is virtually impossible to clean.
• SMAS and other wire spring relay frames: Replace      all SMAS and other wire spring relay frames in line-ups RR2006 – RR2114      due to the inability to effectively clean, mechanical damages, and select      areas of water damage.
• Toll Main Distributing Frame (TMDF): Replace or      retire verticals 1 – 100 due to particulate contamination that is strongly      adhered. Test cleaning indicated that these contaminants could not be      effectively cleaned. TMDF test points in this area also exhibited      corrosion damage.
• All other miscellaneous equipment within 80 feet      of the Washington Street wall should be replaced.

 

Equipment line-ups that are recommended for replacement in the above section are listed in Table 3-1 below. Until this equipment is replaced, Telcordia recommends asbestos level compartmentation with strategic placement of 0.3 micron negative air machines, or detailed HEPA cleaning of this area to minimize cross contamination and/or personnel exposure. Cleaning procedures have been provided under separate cover and are included in Section 11.0 of this report.

 

        

Table 3-1.  Equipment Frame Line-ups Requiring    Replacement
0    Feet to 40 Feet*	40    Feet to 80 Feet*
RR2002   MFT bays	RR2100   SLC Series 5 & D4 Channel banks
RR2003   MFT bays	RR2101   D4 Channel banks
RR2004   MFT bays	RR2102   D4 Channel banks
RR2005   MFT bays	RR2103   D4 Channel banks & SMAS
RR2006   D4 Channel banks	RR2104   D4 Channel banks & SMAS
RR2007   D4 Channel banks & BDFB	RR2110   D4 Channel banks
RR2008   D4 Channel banks & Wescom frame	RR2111   D4 Channel banks
RR2009   D4 Channel banks	RR2112   SLC Series 5
RR2010   D4 Channel banks	RR2113   SLC Series 5
RR2011   D4 Channel banks	RR2114   Telecom Solutions DCD-523 clock frames, SMAS & MTCE Conn
RR2012   D4 Channel banks	RR2116   ADC test panels
RR2013   D4 Channel banks and SMAS	TMDF   through vertical 100
TMDF   through vertical 55

* Distance from Washington Street wall

 

• Additional equipment frames up to approximately 220      feet from the Washington Street wall require detailed external and      internal cleaning including all circuit packs, power and communication      cables to remove the asbestos containing particulate and to restore the      level of cleanliness to a pre-event condition. This equipment includes      line-ups RR2200 through RR2213, RR2300 through RR2306, and RR2400 through      RR2406.  However, to accomplish this      cleaning will require extensive disassembly of equipment frames which must      be performed while limiting service disruption and cross-contamination.      Verizon should compare the costs associated with detailed cleaning,      cabling, cutover, maintenance, service assurance, etc., as well as review      all warranty issues and the interoperability of this equipment with      damaged equipment versus the cost of complete replacement. Equipment that      is not physically damaged but would be difficult to clean is listed below      in Table 3-2.

 

  

Table 3-2.  Equipment    Frame Line-ups Requiring Extensive Cleaning or Replacement
80   Feet –140 Feet*	140 Feet – 170 Feet*	170   Feet – 220 Feet*
RR2200   T1 Repeaters	RR2300   Test points, alarm frames, misc frames	RR2400   D4 Channel banks & SMAS
RR2201   DSX & MDC	RR2301   T1 Carrier	RR2401   D4 Channel Banks & SMAS
RR2202   DSX	RR2302   Assorted muxes, FDF	RR2402   Assorted transport equipment
RR2306   NEC muxes	RR2303   DSX	RR2404   DSX
RR2204   DSX	RR2304   DSX	RR2405   NEC muxes
RR2213   D4 Channel banks & SMAS	RR2305   SLC 5	RR2406   Assorted muxes
TIDF   to vertical 190 (end of frame)

* Distance from Washington Street wall

 

• All external/accessible surfaces of the remainder of      equipment (>220 feet from Washington street) including all power and      communication cables and facility on the 2^nd floor should be      HEPA vacuum cleaned to restore it to a pre-event condition. These      remaining line-ups are listed below in Table 3-3.

 

Table 3-3.   Equipment Line-ups Recommended for HEPA Vacuum Cleaning
RR2217   NEC muxes	RR254   NEC muxes
RR2216   NEC muxes	RR255   SARTS & SMAS
RR2215   NEC RC-28D muxes	RR256   SARTS & SMAS
RR2218   NEC muxes and FDF	RR269   SMAS and cross connects
RR2227   NT muxes	RR270   MTCE and cross connects
RR2228   WE MX3	RR241,   RR242 & RR243  SLC-5
RR235   WE MX3	RR258   & RR268 Test points and relay frames
RR251	RR233   DMS office channel
RR252   Test Panels	RR2240   DMS office channel
RR253	RR2237
RR2238   Versus system	RR246   MTCE frames
RR236   SLC 96 and NEC muxes

 

Photograph 3-1. Severe particulate   contamination of D4 Channel Bank backplane and along card guide in RR2013.05   Bank 3.

The recommendations for network equipment replacement, restoration/replacement, and cleaning is graphically illustrated in the Floor 2 schematic in Section 2.0 of this report.

 

Observations

 

D4 Channel Banks

Particulate had infiltrated the D4 channel banks located on the east (Washington Street) and south (Vesey Street) sides of the second floor when facility walls and windows were breached by falling debris.  This resulted in significant soiling of D4 framework, cabling, circuit packs, unpopulated backplane connectors, and pin fields. Larger, coarse-mode particulate, e.g., concrete fragments, impacted the equipment resulting in areas of abrasion within the electronics and at connectors. The fine-mode particulate resulting from the huge dust clouds associated with collapse of the World Trade Center buildings permeated the D4 channel banks, infiltrated DIP switches, and lodged between electronic component leads and behind surface mount components.  Photographs 3-1 through 3-3 illustrate the severe particulate and debris contamination of the D4 channel bank backplane connectors, card guides, and circuit packs.

 

 

 

 

 

Photograph 3-3.  Severe particulate contamination of a D4   Channel Bank 2FXO plug-in card in RR2009.04 Bank 3.

 

 

Photograph 3-2. Severe particulate   contamination of D4 Channel Bank backplane in RR2013.11 Bank 2.

 

Analyses of four 2^nd floor particulate samples for asbestos via Polarized Light Microscopy (PLM) per New York State ELAP 198.1 Method by a third party laboratory detected asbestos in one sample at less than 1.0% and failed to detect asbestos in the three remaining samples. However, electron microscope observations by Telcordia did identify asbestos fibers on external surfaces of all equipment sampled on the second floor and on internal surfaces of all equipment up to 220 feet from the Washington Street wall. This raises concern with the sensitivity of PLM analysis for detecting fibers in this size range. Telcordia recommends that Verizon EH&S request that Transmission Electron Microscopy (TEM) be used for quantifying asbestos in dust. The laboratory analysis in Section 8.0 of this report describes these findings in greater detail and includes electron microscope photographs of the asbestos fibers.

 

Telcordia laboratory analysis also showed that the particulate contaminant is electrically benign when exposed to elevated relative humidity (RH) up to 98%, i.e., the insulation resistance dropped from ~ 4 x 10¹¹ ohms to ~ 2 X 10⁹ ohms when subjected to elevated RH levels (see Surface Insulation Resistance Test in Section 8.0). This was comparable to a reference “blank” test coupon. However, when hydrated in water, the particulate is corrosive (highly alkaline with a pH of 12-14), and will become electrically conducting.

 

Elevated levels of sulfate were detected on approximately 14% of the D4 channel banks located on floor 2. Telcordia does not attribute this to gas phase transport of products of combustion from fire, but rather to remnant dust that was difficult to remove prior to sampling for chemically absorbed contaminants. This sulfate is not a major concern for short-term equipment operation (See Table 8-3 in Section 8.0).

 

Photograph 3-4.    Chemical   corrosion from water contact with dust on D4 Channel Bank frame RR2102.14.

On-site test cleaning of particulate on frames near the Washington Street wall that exhibited strongly adhered, “caked on” dust and debris using a HEPA vacuum equipped with an ESD brush, and further attempts to clean circuit packs that were removed to our NJ laboratory proved to be ineffective in removing all contaminant. However, the particulate that was less severe in concentration and more lightly adhered on equipment located greater than 80 feet from the Washington Street wall was adequately removed by HEPA vacuum cleaning from the test locations in this area. It should be noted that due to the open architecture of much of this equipment, internal contamination was extensive, and full disassembly is required to access and clean all contaminated surfaces and components, a process that is labor intensive. Detailed vacuum cleaner requirements and D4 cleaning procedures have already been provided to Verizon and are also included in Section 11.0 of this report. Telcordia recommends that all D4 channel banks, associated cabling, framework and hardware located on Floor 2 be cleaned. This will help to minimize personnel exposure to particulate as well as reduce the potential for redistribution and cross-contamination of network equipment and facility. Alternately, for the heavily contaminated frames and damaged equipment that will be taken out of service and replaced, Verizon may opt to provide asbestos level compartmentation in this area and when removed, encapsulate these frames using two layers of plastic sheeting (6 mil minimum thickness) to prevent cross contamination without performing full detailed cleaning.

 

                         

Photograph 3-5. Water damage and      particulate contamination on a plug-in card removed from D4 Channel Bank 3      in RR2009.04.

   In addition to contamination concerns, select D4 equipment frames sustained corrosive damages due to direct contact with water.  The sources of the water were internal firefighting activities, ingress of outdoor water from firefighting activities and/or environmental conditions, and leak water from open standpipes in the building due to pressure build-up during city water system repairs. Photographs 3-4 and 3-5 illustrate corrosive damages to a frame containing D4 and SMAS equipment that was contacted by water. This corrosive damage cannot be repaired and necessitates replacement of the affected frames.

 

Photograph 3-6.  Bent pins on the backplane of a D4 Channel   Bank in RR2011.02 Bank 2 behind 375A power unit.

 

Emergency restoration and cleaning activities, and possibly firefighting activities have resulted in widespread physical damage to D4 backplane pin fields.  Damages were noted on the vast majority of channel banks regardless of vertical location in the frame, i.e., there was no discernable difference in pin field damages to Banks 1 through 6.  Photographs 3-6 and 3-7 illustrate examples of pin field damage.

Photograph 3-7.  Multiple bent pins on the rear of Bank 1,   RR2006.04 D4 Channel Bank frame.

 

 

The physical damages to D4 backplane pins caused multiple electrical shorts and shunts, that resulted in blown fuses on PDUs and widespread channel bank failures. Verizon and Telcordia have tested the following procedure on-site to restore D4 channel bank functionality. Note that this is a labor intensive task, but did clear essentially 100% of the faulty banks that were repaired. Note that the action of straightening bent pins, while clearing the immediate failure mode, will contribute to annealing strain on the pin material, causing the metal to become less ductile due to the dislocation of slip planes within the metal lattice structure.  This will make the pins more susceptible to breaking under shear stresses in the future.

 

 

             

D4 Bent     Pin Straightening Procedure:          Identify channel banks based          on service restoration priority.      Review the fuse status on the          front PDU pack of the D4 channel bank.           If fuse(s) are blown, back out all fuses by twisting out fuse          holders.      Carefully inspect backplane          pin fields and clear (straighten) all bent backplane pins using an          electrically insulated tool.      Reinstall fuse holders,          re-power channel bank, if fuses hold and all alarms clear, log the          status of channel bank.      If fuse(s) blow, repeat steps          1 through 4 since many bent pins are difficult to identify.

 

 

 

 

 

 

 

 

 

 

 

 

 

SMAS and Other Test Wire Relay Frames

Photograph 3-8. Water damage to   relays in RR2116.08. Blue-green electrolytic corrosion and brown oxidative   corrosion  (rust) are both visible.

 

Photograph 3-9.  Physical damage to wire spring relays in   RR2100.01.

Particulate also infiltrated the majority of SMAS and other wire relay frames located near the south and east sides of the second floor, i.e, they are impacted in the same fashion as the D4 channel banks. This resulted in severe soiling of wire spring relays and contacts. Also, select relay frames in lineup RR 2116 were damaged by water, causing electrolytic and chemical corrosion processes.  Emergency equipment cleaning and restoration and possible firefighting activity resulted in widespread physical damage to SMAS relay wire spring wiper arms and contacts. Photographs 3-8 and 3-9 illustrate examples of the contamination, water damages, and physical damages to wire spring relays.

Telcordia developed general vacuum cleaning procedures with wand and brush, compressed air, solvent contact cleaning to test the effectiveness of cleaning wire spring relay frames. Telcordia determined the cleaning was insufficient to remove all particulate, especially that which was embedded within the wire springs.

The particulate was also strongly adhered to the contacts, likely due to operation and high humidity immediately following the event on September 11

Photographs 3-10 and 3-11 provide a comparison between a wire spring relay that was cleaned on-site versus the condition of a wire spring relay from the opposite side of the floor, well away from the dust and water ingress. Following vacuum cleaning, all 2^nd floor wire spring relay contacts would also require burnishing by craft knowledgeable in the process.

Photograph 3-10.  SMAS relay in RR2104.14 after cleaning   attempt. Note remnant particulate.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 3-11.  Non-contaminated SMAS relay in   RR256.05.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2^nd Floor Toll Main Distribution Frame (TMDF)

The 2^nd floor TMDF was contaminated with particulate that had been cursory cleaned. The wire wrap and/or solder connector pins were of sufficient gauge that physical damage did not result from the initial cleaning operation or fire fighting activities. Several vertical blocks in Columns 1 through 35 exhibit more strongly adhered particulate contamination that will require more aggressive cleaning. Corrosion was noted on base mount test points at Columns 6 through 78, necessitating replacement of these test points. The current condition of the frame should not impede short-term network performance, however, replacement of frame elements in damaged facility areas will be warranted. This replacement coupled with corrosion and significant cleaning requirements to restore the frame to a pre-event condition may warrant complete frame replacement. Photograph 3-12 below depicts severe block contamination.

Photograph 3-12. “Caked on”   cement/dust on vertical Column 6 on the 2nd floor distributing   frame after exposure to water vapor.

 

 

Subscriber Loop Carrier (SLC Series 5) & Digital Clock Distribution

 

Particulate infiltrated the majority of SLC Series 5 and Telecom Solutions DCD-523 digital clock distribution frames.  In addition, SLC Series 5 equipment in lineup RR 2112 exhibited significant irreparable corrosion damage to both external backplane pin fields as well as the internal electronics.  Photograph 3-13 is an example of corrosive damage to backplanes, and photograph 3-14 illustrates water trail and electrolytic corrosion behind an integrated circuit chip on an individual circuit pack removed from the SLC 5 system. Based on the widespread water damage and extensive corrosion and contamination, we recommend these frames be replaced immediately. Note, select SLC Series 5 frames and the two Telecom Solution clock frames have already been cut out of service.

 

Photograph 3-13.  SLC Series 5 backplane electrolytic   corrosion damage in RR2112.11.

 

Photograph 3-14. Electrolytic and   chemical corrosion damages on a SLC Series 5 AEK7 circuit pack in RR2112.08.

 

 

Photograph 3-15.  Corrosive damage to Telecom Solutions clock   connector in RR 2114.15.

The Telecom Solutions Digital Clock Distribution frames (DCD-523) in lineup RR 2114 have been damaged by water, resulting in electrolytic and chemical corrosion. Several areas of the frames, including connectors, pins, and fuse/power distribution shelves, exhibited notable evidence of water contact along with corrosive damage. Photograph 3-15 shows an example of corrosive damage to a clock shelf.

 

 

Metal Facilities Terminals (MFTs)

Particulate infiltrated the majority of MFT frames located on the south and east sides of the second floor, resulting in soiling of circuit packs, framework, cabling, backplane connectors, and pin fields. Cleaning of the MFTs in lineups RR 2002, RR 2003, and RR 2004 is required, similar to that described for the D4 Channel banks, but will not be completely effective in removing all residues, especially on the more heavily contaminated frames located near the east wall. There also is evidence of isolated areas of water contact and corrosion.  Therefore, while cleaning will minimize cross contamination and personnel exposure, these frames cannot be fully restored and should be replaced

Multiplexers (muxes)

The NEC, Fujitsu and NT multiplexers on the 2^nd floor were located greater distances from the breach in the wall breaches than the systems discussed above, and therefore sustained more moderate levels of dust contamination. While the levels of particulate contamination on all muxes and supporting optical fiber equipment greater than 80 feet from the wall is not sufficient to cause reliability problems, partial disassembly with detailed internal & external HEPA vacuum cleaning is warranted to remove asbestos containing dust and to restore the equipment to a pre-event condition. It is anticipated that fiber terminations will need to be removed/opened to perform detailed cleaning. Verizon and contractors must be aware of potential service disruptions during cleaning and should clean all fiber terminations prior to reinstallation. Muxes located beyond 220 feet of the damaged east wall should be subject to external HEPA vacuum cleaning only.

 

Miscellaneous Equipment

Miscellaneous frames and ADC DSX shelves located in the 2300 aisle did not exhibit corrosive damage, but did have moderate particulate on exposed surfaces including connectors and test points.  As described in Tables 3-1 through 3-3 and depicted in the Floor 2 schematic in section 2.0, this equipment should be HEPA vacuum cleaned. Although many of the DSX frames are within 220 feet of the east wall, the architecture and lack of plug in circuit packs will enable the DSX and frames to be adequately cleaned using HEPA vacuum cleaning procedures, i.e., there was no evidence of caked on dust.  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 4.0

FLOOR 4

Assessment of Transport Equipment and Toll Distribution Frame

 

 

 

 

 

 

 

Overview: 4^th Floor Transport Equipment

Transport equipment and facility located along Washington Street (east wall) and Vesey Street (southeast corner) of the 4^th floor of the Verizon building sustained significant structural damages, and exposure to water, particulate, thermal stress, electrical stress, and environmental exposure. The chemical content and the nature of the particulate are consistent with our findings for the 2^nd Floor, as are the Telcordia concerns regarding the asbestos content and corrosive potential. Detailed discussion is given to these topics in Section 8.0 of this report.

 

Equipment located in damaged sections of the building that were affected include: Lucent DACS IV, Alcatel 1631, NEC and Fujitsu Muxes, Lucent SLC Series 5, DSC DEXCS cabinets, Nortel DMS Urban lineups, Nortel S/DMS Transport Nodes, DSC/Alcatel Litespan 2000 frames, Applied Innovations Switch shelves, Tellabs Titan 5500 frames, and a toll distribution frame. In addition to direct damages, there are likely electrical damages to circuit packs caused by shorts and shunts that occurred due to water and physical damages in the affected frames, cabling and various interconnect equipment. The presence of asbestos fibers within the particulate found in various equipment sections of the floor will require extensive cleaning to restore to a pre-event condition.

 

Summary of Conclusions

• Toll Distribution Frame (TDF):  The TDF, located along the east wall of      the 4^th floor near the DACS in RR4415, sustained major physical      damage, along with water and particulate damages. Furthermore, the frame      is in an area of the building that requires major structural repair and is      not accessible. This frame should be replaced after building renovations      are completed or replaced with a new frame in a structurally sound and      unimpeded area of the building.
• Tellabs Titan 5500 & ADC      Cross-Connects:  Replace all      equipment in line-ups RR4411 and RR4412 due to significant physical      damage including dents and twisted shelves resulting from ingress of heavy      debris through the east wall. This equipment also sustained severe      particulate entrainment, and exposure to uncontrolled environmental      conditions including temperature and humidity excursions.

• The remaining Tellabs Titan 5500 frames in RR4413      through RR4416: Requires full disassembly, with removal and      examination of each circuit pack, complete cleaning, and functionality      testing due to heavy particulate contamination. Verizon should replace      these line-ups based on contamination, cleaning costs, interoperability      and cabling, practicality of matching new Titan 5500 platform with      older systems, warranty issues, and exposure to uncontrolled temperature      and humidity.
• AT&T SLC Series 5:  Replace all AT&T SLC-5 frames in      line-ups RR4200 and RR4201 due to due to heavy ingress of particulate and      debris, physical damage, and widespread areas of irreparable corrosion      from water.  These frames were      recommended for replacement as soon as possible in prior verbal reports to      Verizon.

–          For temporary SLC Series 5 service restoration, excluding water-damaged SLC frames RR 4200.07 through RR 4200.15, we suggest that damaged backplane pin fields be repaired by straightening the pins, and if necessary, faulty packs be replaced. Emergency restoration and possibly firefighting activities damaged multiple backplane pin fields resulting in electrical shorts, shunts, and other failures, similar to that exhibited on the 2^nd floor D4 channel banks.

• Replace all transport equipment frames in line-ups      RR4000-RR4019 due to heavy water damage, physical damage, particulate      entrainment and environmental exposure.       This equipment includes Nortel DMS-Urban Switch frames and S/DMS      Transport Nodes, Alcatel Litespan-2000, Fujitsu and NEC muxes, Lucent DACS      II and DACVS IV, DSC DEXCS cabinets, and miscellaneous transport/transmission      frames. Continuing network troubles are occurring from these equipment      lineups that remain online.

 

The equipment frame line up numbers and corresponding types of equipment requiring replacement are listed below in Table 4-1.

 

  

Table 4-1. Equipment     Requiring Replacement
RR4000   and RR4001 Nortel DMS-Urban
RR4002   Alcatel (DSC) Litespan-2000
RR4003   Nortel S/DMS Transport Nodes
RR4004   Fujitsu FLM-150 muxes
RR4005   & RR4006 DSC iMTX
RR4007   Fujitsu FLM-150 muxes
RR4008   NEC muxes
RR4009   Fujitsu muxes
RR4010   NEC FD1840A muxes
RR4011   Muxe
RR4012   NEC RC-28D muxes
RR4013   through RR4015 DACS IV
RR4016   DSC CS1VL and DACS II
RR4017   & RR4018 DSC DEXCS
RR4019   DSC Master Timing Shelves, DEXCS K02cabinets
RR4200   AT&T SLC Series 5.
RR4201   AT&T SLC Series 5
RR4411   through RR4416 Tellabs Titan 5500 & ADC Cross connects

 

• Tellabs Titan 5500 located in lineups RR4114, 4115      and RR4308, 4309 and Alcatel DACS K32 & K44 in RR4301 to RR4305 exhibit      moderate to heavy dust accumulation due to ingress of particulates through      broken windows in the south wall of the 4^th floor.  No water or physical damages were      observed on this equipment. This equipment should be completely cleaned,      internally and externally, to restore it to a pre-event condition. This      will be labor intensive and disruptive to service due to the level of      disassembly of systems required for full cleaning. Verizon should compare      the costs associated with cleaning this equipment with the costs of      replacement. Furthermore, it is anticipated that equipment located in RR      4300 lineups will be exposed to severe vibration and may be inaccessible      during times of facility repair. Equipment that is not physically damaged      but would be difficult to clean is listed in Table 4-2.

 

  

Table 4-2. Equipment    Recommended for Internal & External Cleaning or Consider Replacement
RR4114 Tellabs Titan 5500 DACS
RR4115 Tellabs Titan 5500 DACS
RR4100 and RR4101 Fiber Cross Connects
RR4301 through RR4310 Alcatel 1631, 1633, Tellabs   Titan 5500
RR4102-4112 Assorted Titan 5500, Alcatel, iMTNL   frames

 

• The remainder of the equipment on the fourth floor      sustained no water damages or corrosion, but minor asbestos contamination      was noted. Equipment includes fiber optic distribution shelves, cross      connects, multiplexers, test frames, D4 channel banks, peripheral data and      computer equipment, etc. This equipment should be HEPA vacuum cleaned on      all accessible/external equipment and facility surfaces to remove      particulate that had been distributed from damaged areas of the building      to mitigate risks form the presence of asbestos containing particulate.
• Although a strong smoke odor had originally been      present throughout the fourth floor transport spaces during our initial      visits on September 12 through 14, there is no evidence of impact from gas      phase transport of contaminants from the smoke/fire gases onto equipment.      (See Chemical Analysis Section 8.0 of this report)

 

Refer to Floor 4 schematic in Section 2.0 for graphical representation of cleaning and replacement requirements.

 

Observations

 

Equipment that was directly impacted by the attack was located in close proximity to the breaches that penetrated the Washington Street wall of the building and the broken windows in the southeast corner. The east wall damage was severe on the fourth floor, with heavy debris entering the space in some areas as shown in Photograph 4-1. This debris caused significant physical damage to select equipment frames.  These included the Tellabs Titan 5500 shelves, which were severely bent and dented, and the toll distribution frame (TDF) that was partially toppled over and severely damaged.  These spaces also suffered heavy water damage to numerous equipment line-ups, along with severe particulate entrainment within equipment.

 

Photograph    4-1.   A breach in the Washington Street wall of   the 4th Floor near frame and DACS room.

 

Analyses of three 4^th floor particulate samples for asbestos via Polarized Light Microscopy (PLM) per New York State ELAP 198.1 Method by a third party laboratory detected asbestos in one sample at less than 1.0% and failed to detect asbestos in the two remaining samples again raising concerns with the sensitivity of the PLM test. Additional electron microscope observations by Telcordia identified asbestos fibers to various degrees within 24 remaining samples collected from the 4^th floor. The laboratory analysis Section 8.0 of this report describes these findings in greater detail and includes electron microscope photographs of the asbestos.

 

Elevated levels of sulfate were detected on approximately 9% of the equipment located on floor 4. Telcordia does not attribute this to gas phase transport of products of combustion from fire, but rather to remnant dust that was difficult to remove prior to sampling for chemically absorbed contaminants. This sulfate is not a major concern for short-term or long-term equipment operation (See Table 8-4 in Section 8.0).

 

In addition to contamination concerns, multiple equipment frames sustained corrosive damage due to direct contact with water.  The source of the water was both internal firefighting activity, ingress of outdoor water from firefighting activity and/or environmental conditions, and running water from open standpipes in the building due to pressure build-up during city water system repairs. Photographs throughout this section illustrate water damages, where applicable.

 

Toll Distribution Frame

The toll distribution frame (TDF) is located along the east wall of the fourth floor near Tellabs Titan 5500 frames in RR4415. Structural members from the collapsing WTC 7 caused major damage at the east wall and caused considerable physical damage to the TDF with one line-up partially collapsing as shown in Photograph 4-2.  Severe particulate contamination was observed throughout the TDF. The particulate fully covered the entire frame and its components with strongly adhered contaminant that will be difficult to clean effectively. The particulate also contains asbestos fiber. Water damage observed on the frame included areas of minor pin corrosion. The TDF is located in an area of the building that will require significant structural repair activities, and will require removal in order to facilitate these building repairs.  In addition, the area is inaccessible due to safety concerns. Although the scope of the of the repair work has not been finalized, we expect that it will not be feasible to retain the TDF, and we recommend that a new TDF be installed in a sound area of the 4^th floor and the current TDF be removed.

Photograph 4-2. Partial collapse of the toll distribution   frame located along the east wall of Floor 4.

 

 Tellabs Titan 5500 Frames and ADC Cross-Connect Shelves

Several line-ups of Tellabs Titan 5500 frames run parallel with the east wall in RR4413, RR4414, and RR4415.  RR4412 contains ADC cross connect shelves, parallel with the Tellabs equipment.  Severe damage to the east wall led to debris intrusion into the space causing physical damage to select shelves, along with particulate contamination and environmental exposures.  In addition, the area is at times, inaccessible due to safety procedures that are invoked whenever recovery activities at the adjacent WTC 7 site occur.  This situation is likely to continue until the disaster recovery activities at WTC 7 are completed. Photographs 4-3 and 4-4 illustrate physical deformation of Titan 5500 and ADC cross-connect shelves.

Photograph 4-3. Physical damage to ADC cross-connect   shelves in RR4412.

 

Photograph 4-4. Physical damage to Tellabs Titan 5500   shelf in RR4413.01.

 

 

 

 

 

AT&T SLC Series 5

Two line-ups of AT&T SLC 5 frames were located in Aisles 4200 and 4201 running parallel with the east wall.  All SLC 5 frames in 4200.09 through RR4200.15 sustained significant irreparable electrolytic corrosive damage resulting from direct water contact. Electrolytic corrosion was observed on backplane pins on the rear, front side test points, plug-in cards, and on the fuse bays.  In addition, as in the case of the 2^nd floor D4 backplanes, many backplane pins had been bent during firefighting and/or emergency restoration and cleaning activities that occurred in this area. Straightening the individual bent pins will allow temporary service restoration in banks with no water or pack damage, however, this measure should be undertaken for temporary restoration only; replacement of the equipment will eventually be required. Photographs 4-5 through 4-7 illustrate examples of the SLC 5 damage on the fourth floor. In addition to the SLC 5, the BDFB in RR4201.16 serving the transport equipment sustained corrosion of the copper buss as it also was within the area of significant water ingress.

Photograph 4-5. Water damage to SLC 5 in front of AUA 38   in system 174 in RR4200.10.

Photograph 4-6. Water damage to a SLC 5 AUA38 circuit pack   in RR4200.14.

 

 

Photograph 4-7.    Bent backplane pins in SLC5 RR4201.09 in system 353.

 

Additional Transport Equipment Damages

Oriented perpendicular to the SLC 5 frame line-ups are transport equipment line-ups RR4000 through RR4019.  These line-ups extend along the Washington Street wall of the building.  The damage and breaches at the exterior wall resulted in several damage mechanisms on this equipment that include severe particulate and debris entrainment, physical damage, environmental exposure, and extensive water damage from both the outdoors and from fire fighting activity.  Fire hoses were used in this space to direct water out of the east wall windows onto World Trade Center Building 7. The transport equipment that requires replacement was outlined in Table 4-1 in the summary of conclusions at the beginning of this section. Descriptions and photographs of the damage are presented here. 

 

RR4000-RR4001 Nortel DMS Urban

Photograph 4-8. Heavy water residues, corrosion, and   contamination on 4th floor DMS Urban switch.

The Nortel DMS Urban Switch comprised the first two line-ups from the north side of the room and consisted of bays 00-10 in RR4000 and bays 00-07 in RR4001.  The switch sustained very heavy dust and debris contamination and extensive water damage including both corrosion of electronics and rusting of framework.  Additionally, the water and high humidity caused much of the heavy dust to adhere “cake on” to the equipment surfaces, making it impractical to fully clean. The areas of corrosion were evident on backplanes, within connectors, on fuse shelves and frame hardware and on circuit packs bridging between and underneath components. Photographs 4-8 and 4-9 illustrate the damage. This switch is unrecoverable and must be replaced.

Photograph 4-9. Water trails and corrosive residues   between solder points and connector pins in the 4th  floor DMS Urban switch.

 

RR4003 Nortel S/DMS Transport Node

The Nortel S/DMS fiber optic equipment is located in RR4003 Bays 00 through 09. Like the DMS Urban, this equipment also suffered severe particulate contamination along with corrosion and rusting due to water damage. Very heavy dust covered numerous circuit packs and was caked onto individual component leads and connector contacts.  Water damages were observed with corrosion on the rear power shelves and upper shelf exposed metals.  Photograph 4-10 shows heavy dust on a NT4K52FA processor card in Bay 08 and Photograph 4-11 shows extensive water damage in Bay 04. Due to the level of damage, difficulty cleaning, and exposure to extreme environmental conditions, this equipment must be replaced.

 

Photograph 4-10.    Severe particulate contamination of a NT4K52FA board in RR4003.08.

 

                                     

Photograph 4-11. Corrosive damage to Nortel S/DMS transport       node in RR4003.04.

     
RR4004A Fujitsu FLM-150 ADMs

Fujitsu muxes sustained heavy contamination as well as multiple areas of corrosion and hardware rusting.  As with the other equipment described previously, the proximity of this line-up to the damaged wall of the building exposed it to dust, debris, water, uncontrolled temperature, and high relative humidity. It should be noted that this equipment was operational and that internal circuit packs could not be inspected without provoking service disruptions. The architecture of the shelves and the presence of water in upper portions of the shelves make it highly likely that additional corrosive damage is present on internal electronics. Photographs 4-12 through 4-15 illustrate the water damage to these frames.

Photograph 4-12. Contamination and corrosion of connector   and chassis surfaces and associated hardware in RR4004A.02.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 4-13. Contamination and corrosion on EMI   shielding of a Fujitsu FLM-150 unit located in RR4004A.07.

 

 

 

 

 

 

 

Photo 4-14. Corrosion and contamination in Fujitsu FLM-150   ADM shelf in RR4004A.02.

 

 

 

 

Photo 4-15. Rusting of Apex fuse panel above FLM-150 in   RR4004A.04.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

RR 4010 – RR4012 Muxes

NEC FD-1840 Muxes in RR4010-RR4011 exhibited heavy dust contamination, and areas of corrosion.  The frames nearest the end of the line-up located closest to the wall breach exhibited greater amounts of contamination and abrasive bulk debris entrainment.  Photographs 4-16 and 4-17 illustrate this contamination. Line-up RR4012 contains NEC RC-28D muxes with similar contamination and corrosion concerns.

 

Photograph 4-16. Heavy dust infiltration in NEC MUX 3 in   RR4010.12.

 

 

Photograph 4-17. NEC FD1840 mux in RR4010.09 particulate   contamination on internal circuit packs.

 

RR4014 – RR4015 DACS IV

The Lucent DACS IV systems sustained heavy dust contamination and exposure to moisture and extreme environmental conditions.  Evidence of water damage was observed on five (5) DACS frames located in RR4015.12 through RR4015.16. The fully populated configuration of the systems makes it impossible to look for water damage to internal electronics without complete disassembly. Very heavy particulate entrainment and bridging across integrated circuit chips was noted on numerous AKM66 and AKM68B packs as well as within main controller bay.  Photographs 4-18 and 4-19 illustrate some of the damages.  An additional concern is thermal stress due to the closed configuration and high heat load (200 watts/square ft) of this equipment.  This equipment should be replaced.

Photograph 4-18. Heavy dust covering internal circuit packs   and bridging IC chip leads.

 

Photograph 4-19. Heavy build-up of dust and debris on   horizontal surfaces.

 

RR 4016 DACS II and DSC CS1VL

Photograph 4-20.    Water damage and corrosion to rear connector hardware and chassis on   DACS II 1SX frame in RR4016.07.

This DACS II systems also sustained severe particulate contamination along with numerous areas of water damage. The water damage consisted of rusting of hardware such as chassis, threaded connector bolts, and hex nuts on individual circuit packs and framework.  Water trails and various residues both from water and particulate were evident emanating from below circuit board mounted components. Similarly DSC frames in the same line-up also exhibited various damages associated with corrosion and heavy contamination/abrasion from coarse mode particulate.  Photographs 4-20 through 4-22 illustrate examples of irreparable corrosive damage in RR4016.  It is recommended that this equipment be replaced.

 

Photograph 4-21. Corrosive damage and contamination on DSC   CS1VL Island 2.

 

Photograph 4-22. Corrosive damage and contamination to DSC   CS1VL electronics.

 

 

RR4017 DSC DEXCS Cabinets

The DEXCS cabinets and miscellaneous transport frames in aisle RR4017 sustained physical damage to frames closest to the east wall, as well as several areas of water damage.  All had significant dust entrainment. All DEXCS cabinets in RR4017 should be replaced due to reliability concerns.

 

The remainder of the equipment in line-ups RR4000 to RR4019, which includes assorted cross-connect, fiber optic, DSX, and various multiplexers all sustained similar extreme environmental exposures with areas of water and physical damage with significant particulate contamination.  Although it would be possible to salvage some individual circuit packs, it would require full disassembly, close inspection of hundreds of circuit packs and connectors, detailed cleaning of asbestos-containing dust, and re-certifying and testing functionality in a test set-up offsite.  Based on Telcordia experience with restoration of electronic equipment, this procedure would likely not prove cost effective. 

 

A separate equipment area located along the center of the south (Vesey Street) wall was not exposed to the main breach of the east wall, but was subject to massive particulate cloud which entered through blown-in windows facing the World Trade Center Towers.  Equipment in this space consists of Alcatel 1631/1633 and Tellabs Titan 5500 located in RR4301 through RR4310, fiber optic cross-connect distribution frames in RR4100 through 4101, and miscellaneous transport frames in RR4102 through 4112. While this equipment was intact and functioning, it exhibits moderate to heavy particulate contamination that includes asbestos fibers.  It also was exposed to harsh environmental conditions, high heat loads, and unknown levels of shock/vibration. The proximity of the equipment to structurally damaged areas of the building that require repair will subject the equipment to additional vibration concerns during the restoration, and will impede access to equipment for service issues. Verizon also should consider the added expense for additional support during building repair. While it is technically feasible to restore this equipment, Verizon should consider the practical implications for interim equipment space access, costs of disassembly and cleaning, and risk of physical damage associated with anticipated building repair. 

 

The remainder of the network equipment on the fourth floor is located along the west and north walls to the northwest corner of the building. This space requires HEPA vacuum cleaning of all accessible external equipment and facility surfaces.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 5.0

FLOOR 7

Assessment of 5ESS and DMS-100 Switch Rooms, and Ancillary Spaces

 

 

 

 

 

 

 

 

Overview: Floor 7 Switch Rooms

The seventh floor of 140 West Street contains two Lucent 5ESS switching systems (DS-1 and DS-2) sharing a single large room along the Washington Street and Vesey Street sides of the building. In addition, one Nortel DMS-100 switch (DS-3) is in a dedicated, compartmentalized switch room along the Barclay Street (north wall) of the 7^th floor.  Like floors 2 and 4, the seventh floor sustained severe damage along Washington Street (east wall) and Vesey Street (south wall). Several holes in the east wall along with partial floor slab collapse resulted in massive particulate contamination, severe physical damage to switch frames, and prolonged exposure to uncontrolled environmental conditions.  Water damages were not a factor in 7^th floor equipment.  In one portion of the 5ESS switch room, a collapsed floor slab resulted in a group of 5ESS frames falling through the floor, suspended only by the overhead cabling serving the switch modules.  Telcordia letter 4337/01-198 documented these damages, identified emergency restoration issues, and provided recommendations for short-term recovery based upon our observations at the site on September 12, 2001. This section provides detailed restoration recommendations for the switch rooms to resolve short-term and explain long-term impacts from this event. A 7^th floor mechanical room located on the Vesey Street side that houses a Liebert air handler, also sustained severe physical damage to the external wall.

 

As with the network equipment on the 2^nd and 4^th floors, the particulate contaminant is comprised primarily of pulverized concrete, wallboard, and plaster co-mingled with fibers that include asbestos.  The chemical content and the nature of the particulate are consistent with our findings for other floors, and are fully described in Section 8.0. 

 

Summary of Conclusions

 

Lucent 5ESS Switch (DS-2)

• DS-2 is located in the southeast corner of the 7^th      floor, a location that sustained major structural damage.
• Replace DS-2 due to direct physical      damage and deformation to switch frames, severe particulate ingress and      abrasion, excessive shock and vibration, uncontrolled elevated humidity      and temperatures, and future inaccessibility during facility restoration      and logistical problems during all building structural repair work. Note,      elevated relative humidity resulted in slight chemical corrosion of      chromated zinc framework as evidenced by discoloration of framework in the      SM101 aisle along the Vesey Street wall.

 

Lucent 5ESS Switch (DS-1)

• DS-1 is located in the northeast section of the 7^th floor within the same open space as DS-2, but is located farther from the significant wall breaches and collapsed slab areas than DS-2.  However, numerous broken windows and the open floor plan permitted particulate ingress and exposure to relative humidity and temperature extremes. Also, shock and vibration concerns are raised.

• No physical damage was sustained to the switch      frames, cables, or support structures of DS-1.
• The particulate contamination of DS-1 can be removed      by detailed external and internal HEPA vacuum cleaning. The level of      cleaning will require removal of all circuit packs and some frame      disassembly. The cleaning would also require full asbestos abatement      procedures and verification of asbestos removal.
• Verizon must consider several complicating factors      that may make full replacement of the DS-1 switch more cost effective      than extensive particulate cleaning.       The cleaning would necessitate periods of service disruption.   The inaccessibility of the switch for      service during facility repair of the east wall breaches and associated      floor damage will complicate the logistics of maintaining the equipment      during the restoration period and will result in periods where the switch      will be inaccessible.  The extensive      repair activities could also result in vibration induced outages to the      switch during the construction. Finally, the costs of performing the      complete internal and external cleaning and testing of the switch as well      as warranty concerns should be evaluated against the replacement cost of      the switch.

 

Nortel DMS-100 Switch

• The DMS-100 space and associated MAPS room are compartmentalized and located on the Barclay Street 9 (north side) of the building.  No exterior wall breaches occurred in this space. The switch exhibited substantially less contamination than the 5ESS switches. All accessible external switch surfaces as well as facility surfaces including cable racks, light fixtures, tabletops and windowsills should be HEPA vacuum cleaned as a precaution should trace asbestos fibers be present via cross-contamination.
• After the loss of primary power and HVAC system on September 11, the switch continued to run on battery for several hours.  This resulted in temperatures exceeding 100 degrees F in this space. Additionally, after restart, high temperatures were also expected to have occurred.  However, the lack of frame filter loading and component blanketing with particulate should not have resulted in unusual switch temperatures, i.e., based on reports the DMS was not exposed to temperatures above its intended design criteria.
• The DMS-100 can be certified for long-term operation after complete diagnostics including simulation of full capacity testing. This is warranted to ensure that elements of the switch were not compromised and/or to identify any damaged or dysfunctional circuit packs.

 

Collocation, Storage and Mechanical Room

• One mechanical room located on the south wall of the facility sustained physical damage.  Once physical damages are repaired, this room should be fully HEPA vacuum cleaned, and the Liebert Air Handling Unit (AHU) should be disassembled and cleaned, and filters within the unit should be replaced.   
• All other spaces, including a large collocation room along the west wall, storage areas, and mechanical rooms located in the north and west sides of the building, sustained no direct damage with the exception of two broken windows in the collocation space. These areas should be HEPA vacuum cleaned to remove particulate contamination from all accessible facility and equipment surfaces. These surfaces include light fixtures, cable racks, framework, stored equipment such as cable reels and boxes, and the floor.

 

Observations

Lucent 5ESS Switch (DS-2)

The 5ESS switch DS-2 sustained the most serious damages on the 7^th floor.  The proximity of the switch to major structural damage to the building’s east wall and partial collapse of the floor slab resulted in extensive physical damages to frames and cable racks.  Breached walls and broken windows also caused severe contamination of the equipment. Furthermore, the switch was open to the outdoor environment for an extended period of time.   Photographs 5-1 through 5-7 below present the condition of DS-2 switch space as of September 12, 2001.

                         

Photograph 5-2. Blown out wall and      rubble piled up on 5ESS switch frames.

                          

Photograph 5-5. Blown out east wall      and debris in 5ESS switch room.

                            

Photograph 5-1. East wall and partial floor collapse in 5ESS      switch room.

                              

Photograph 5-4. 5ESS switch modules falling through collapsed      floor slab and suspended by cabling.

            

Photograph 5-3. Deformed steel building frame and collapsing cable rack hardware.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 5-6. Debris and particulate   contamination on 5ESS    switch module   frames along Vesey Street that did not sustain physical damage.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                         

Photograph 5-7.  Damage      to 5ESS switch room raised floor, switch modules and overhead cabling.

    

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

As shown in the above photographs, the damages within the 5ESS switch space included twisted cable racks, deformed or collapsed building structural elements, physical damage to switch modules, impacts from heavy debris, massive particulate contamination, and exposure to outdoor environmental conditions. Photograph 5-1 illustrates that the Washington Street wall was blown onto 5ESS switch modules, compromising the floor slab and structural integrity of the frames. Photograph 5-2 shows large amounts of heavy debris piled up on the switch frames and the twisted steel and structural elements that further damaged the 5ESS.  Photograph 5-3 illustrates collapsed cable rack serving the 5ESS.  This damage resulted in stresses to the connecting equipment and the potential of electrical shorts of power cable to framework causing a significant fire hazard.  It was recommended by Telcordia that extensive clearing of this cable would be required for safety.  The partial collapse of the floor slab resulted in several switch modules breaking through to the sixth floor below. These frames were prevented from crashing to the floor below by the signal cables that were serving the respective switch modules.  Photograph 5-4 shows this floor collapse in greater detail. Other switch modules suffered physical impact as large debris crashed into the frames, and floor and ceiling support became deformed. Photographs 5-5 – 5-7 further illustrate the extent of contamination, damages, and debris within the 5ESS switch room. 

 

The physical damage to switch modules and power/signal cable may also have resulted in electrical damages to interconnected switch modules that were not physically damaged.  The circuit pack integrated components, e.g., logic circuits, are sensitive to electrical overstress conditions, resulting in loss of functionality of components without any visible damages.  Furthermore, the dedicated customer line processing of surviving 5ESS frames will likely impede diagnostics, as atypical operations and failures may not become known until certain traffic load patterns are experienced.

 

Switch frames in line-ups at greater distance from the damages did not exhibit obvious physical damages, but were severely contaminated with abrasive particles and blanketed with a thick layer of dust.  Severe contamination, including coarse concrete fragments, was present on all equipment surfaces, including frame filters, fans, card guides, open connectors, and between backplane pins.  Moderate dust was present on circuit packs. The contamination was pervasive throughout all switch modules and support equipment in the space.  Specific concerns with the contamination include the presence of asbestos fibers and associated occupational health risks; abrasion and damage as particulate is entrained between connectors and electronic component leads; thermal blanketing, and the potential for corrosion if the particulates absorb moisture. The circuit packs were partially protected from the full amount of available particulate by the frame filters that were fully loaded “clogged” with dust and debris.  Photograph 5-8 shows a uniform layer of dust covering an internal switch frame surface.  The bulk particulate and debris cleaned up fairly well, but areas of the 5ESS line-ups near the Vesey Street wall exhibited particulate that was more strongly adhered, i.e., the particulate was not adequately removed by vacuum cleaning.  The exposure to high humidity from the exterior environment likely contributed to the adhesion of the particulate onto metal switch surfaces. Even after cleaning, notable discoloration and caked on contamination was observed on fan grills, and dust internal to the switch modules remained.  Photographs 5-9 through 5-11 show the current contamination concerns.

                         

Photograph 5-9. Particulate adhered to      SM fan grills.

                            

Photograph 5-8.  Heavy      dust covering all exposed equipment surfaces within the 5ESS switch      modules.

  

 

Photograph 5-10.  5ESS contamination remaining within switch   modules after cleaning. Note also the slight discoloration of the zinc   chromate metal finish.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 5-11.  5ESS contamination remaining within switch   modules after cleaning.

 

 

Analyses of six 7^th floor particulate samples (Section 8.0, Table 8-1) for asbestos via PLM by a third party laboratory detected asbestos at a level over one percent in two samples (7^th Floor 5ESS Space, 9/13; 7^th Floor Collocation Room, 9/13), less than one percent in one sample (7^th Floor Mechanical Room, 9/13), and did not detect asbestos in three samples. Additional electron microscope observations by Telcordia identified asbestos fibers within all samples collected on the 7^th floor.  Section 8.0 of this report describes these findings in greater detail and includes electron microscope photographs of the asbestos.  The majority of the particulate is not dense and easily becomes airborne when disturbed. Telcordia recommends that all internal and external 5ESS frames, associated cabling, framework and hardware located on Floor 7, including under raised floor be cleaned. See 5ESS cleaning procedures in Section 11.0. This will help to minimize personnel exposure to particulate as well as reduce the potential for redistribution and cross-contamination of network equipment and facility. Alternately, Verizon may opt to provide asbestos level compartmentation in the 5ESS space and when the switches are removed, encapsulate all frames using two layers of plastic sheeting (6 mil minimum thickness) to prevent cross contamination without performing full detailed cleaning.

 

Analyses of surface residue via ion chromatography revealed no elevated levels of ionic contaminants (See Table 8-6 in Section 8), therefore, ionic contamination of 7^th network equipment is not a concern.

 

While the frame filters were completely loaded “clogged” with particulate and debris, the frames remained operational on battery reserve after loss of primary power and subsequent loss of environmental control. As such, the switch modules were subjected to room temperatures that exceeded exceeding 100 degrees F. The impacts of the elevated temperatures were exacerbated by the lack of fan airflow due to the clogged frame filters.  Additional internal heat dissipation difficulty was caused by the thermal blanketing of electronic components with dust. The thermal concern is addressed in Section 10.

 

Telcordia strongly advises that all remaining 5ESS switch modules comprising DS-2 be replaced.  Replacement is warranted due to serious concerns with the reliability and full functionality based on the following damages:

• Physical Damage
• Internal, asbestos containing particulate contamination that may be impractical or impossible to fully abate
• Exposure to uncontrolled environmental conditions, including high relative humidity, water vapor generated by firefighting activities at the World Trade Center Buildings, and elevated temperatures due to loss of environmental control
• Inaccessibility of the 5ESS space during upcoming structural renovation
• The potential need to remove some frames to facilitate slab repair
• Potential vibration and shock damage during structural renovation
• Cable risers feeding this switch are located in close proximity to where demolition will likely occur prior to restoration.

 

Restoration costs, which include 100% inspection and testing of circuit packs, 100% inspection of connectors and pin fields for corrosion, replacement of select frames due to corrosion, detailed cleaning, and the unknown cumulative impacts from the event on long-term reliability, would not in Telcordia Technologies view, justify salvaging this switch. In addition, warranty concerns and switch inaccessibility by craft and the potential for additional physical damages and service disruptions due to upcoming major structural repairs to the building, negate long-term operation of DS-2.

 

DS-1

The 5ESS switch designated DS-1 is located in the northeast corner of the building in the same space as DS-2.  DS-1 did not sustain the physical damage seen in DS-2 since it was located away from the major structural damages. However, since the space was open between both 5ESS switches, DS-1 did sustain significant particulate and debris contamination and exposure to untempered outdoor conditions.  As with DS-2, elevated temperatures associated with its operation in an uncontrolled environment with severely loaded frame filters is also concerning.  Refer to section 10.0 for details on thermal concerns under varying heat loads.

 

The detailed internal and external cleaning of DS-1 is expected to be labor intensive, requiring removal of all circuit packs, partial frame disassembly, and the potential for extended loss of service within frames when they are being cleaned. This level of cleaning and restoration is required to alleviate asbestos concerns, and to restore the switch to a pre-event condition.  The restoration must also take into account building renovation and construction activities in close proximity to the switch; these will limit accessibility to the switch and pose a significant risk for service interruptions and physical damages.

 

While it is technically feasible to restore DS-1, Verizon must consider all warranty and service impacting issues, the unknown cumulative impacts from the event on long-term reliability as well as the projected costs to accomplish the restoration.  It is likely that switch replacement would be a more cost effective alternative.

 

DMS-100

The Nortel DMS-100 switch is located in a compartmented switch room on the north side of the 7^th floor.  The room sustained no physical damage to the exterior wall, no broken windows, and no significant contamination of particulate.  The cable racks and floor were in good condition, frame filters were not heavily soiled, and passively cooled frames had no abnormal indication of contamination.  Some light dust was present on horizontal surfaces but it could not be distinguished from dust accumulations typically observed on equipment of this age.  As such, Telcordia does not believe that external switch and facility cleaning is mandatory for this space, but recommends that Verizon consider HEPA cleaning of all accessible external switch and facility surfaces as a precaution should trace asbestos fibers be present via cross-contamination.

There was no indication of water ingress into this space. Also, the location of the DMS-100 switch room should maintain adequate separation from the building renovation construction work, i.e., there is minimal concern with inaccessibility of the space or physical damages associated with the construction.

 

The primary concern with the switch is the high temperatures experienced in this space after the loss of environmental control.  As a fully sealed room with all windows intact, we expect that the temperatures exceeded 110 degrees F.  At the time of our initial visit to the site on September 12, 2001, the temperature was over 90 degrees F in the DMS-100 room, approximately 18 hours after the power to the switch was lost.  Although temperatures were high, unlike the other switches in the building, the frame filters were not clogged and thermal blanketing of the electronics by particulate did not occur. While we expect there will be some fall out of packs that were compromised by elevated temperature, we believe that the DMS-100 can be certified for long-term operation after diagnostics simulating full load testing is completed to identify any damaged or dysfunctional circuit packs.

 

Mechanical Rooms, Collocation Room and Storage

                         

Photograph 5-12. Mechanical room containing Liebert air handler      sustained heavy damage to wall.

   One mechanical room located adjacent to the 5ESS room along the Vesey Street wall sustained serious structural damage that resulted in significant debris and particulate ingress.  Photographs 5-12and 5-13 illustrate the damages and contamination.  This room requires extensive repairs and the Liebert Air Handling Unit and space should be subject to cleaning of all external and internal surfaces.  Associated ductwork should be evaluated and cleaned as necessary.

 

 

 

 

 

 

 

 

                         

Photograph 5-13. Liebert Air handler      unit in 7th floor mechanical room.

  

 

 

Two additional mechanical rooms on the north side of the floor were not impacted. If not already accomplished, filter replacement, internal duct inspection, and cursory cleaning of any remnant particulate residues should be completed in the HVAC systems contained in this space.

 

A large collocation and storage area on the west side of the floor sustained moderate particulate ingress through two broken windows.  This space requires full HEPA vacuum cleaning of all accessible surfaces, including the tops of cable racks, lighting fixtures, equipment cages, and ductwork.  Furthermore all stored equipment boxes, cable reels, hardware, etc must be HEPA vacuum cleaned.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 6.0

FLOOR 9

Assessment of DMS-100 Switch Room, Power Room, Cosmic Frame, Voice Mail and Assorted Collocation and Storage Spaces

 

 

 

 

Overview: Floor 9

Network equipment spaces on the 9^th floor consist of a Nortel DMS-100 switch room, a Power (battery) Room, and a Main Distributing Frame (Cosmic frame). As in the case for floors 2, 4, and 7, the 9th floor sustained damage along the Washington Street wall and a portion of the Vesey Street wall. Multiple windows were broken in the DMS-100 switch room and the Cosmic frame space as debris from outside crashed through. This caused severe particulate contamination of the switch, cosmic frame, and facility. In addition, larger fragments of concrete and debris entered the DMS space. The loss of air conditioning and open windows subjected the DMS switch equipment to untempered air with uncontrolled temperature and high relative humidity and moisture from firefighting activities.  Telcordia letter 4337-01-198 documented our initial assessment and provided emergency restoration issues for short-term recovery of this equipment.  This section provides detail on restoration recommendation for the DMS-100 room, Cosmic frame, Power room, mechanical room, voice mail space and collocation spaces for both short term and long-term impacts from this event.

 

Summary of Conclusions

 

Nortel DMS-100 Switch

• The DMS-100 space and adjoining MAPS room are compartmentalized and located in the southeast corner of the ninth floor.  Blown in windows on both the east and south walls resulted in severe particulate accumulation throughout the DMS-100 switch frames, cabling, and facility surfaces including under raised floor. The particulate was found to contain asbestos fibers.  
• Contamination was heavy within frames, internal shelf surfaces, on circuit packs and circuit pack electronic components, and in open connectors and card guides.
• The presence of asbestos containing particulate contamination within the DMS-100 requires the removal of all circuit packs for HEPA vacuum cleaning, along with partial disassembly of frames to adequately access and abate.
• Asbestos level HEPA vacuum cleaning must also be conducted under the raised floor.
• Complicating the cleaning is the anticipated lack of accessibility to the DMS-100 switch during major building renovations, including structural column removal/replacement and façade repair within the switch space. 
• The construction activities in the space also will likely expose the switch to vibration levels in excess of the design criteria for the DMS-100 and details to minimize vibration must be undertaken.
• After the loss of emergency power and the HVAC system, the switch continued to run on battery for several hours.  This resulted in temperatures exceeding 115 degrees F in the space with elevated switch temperature that was further exasperated by loaded frame filters and thermal blanketing of circuit pack components.  Additionally, after restart high temperatures also were reported to have occurred.  
• While the DMS-100 can technically be cleaned, Telcordia strongly recommends that Verizon replace the DMS-100 due to the obstacles presented above, namely the high costs of restoration work, loss of service during remediation, interoperability issues, warranty issues, potential for additional damages, and uncertain logistics.
• The MAPS room and storage room adjacent to the DMS-100 switch room should be fully HEPA vacuum cleaned.

Cosmic Frame

• The distributing frame and surrounding support and facility structures sustained severe contamination with asbestos containing particulate.
• The initial cleaning was effective to remove the bulk of the dust and debris throughout the frame, however, additional detailed cleaning is required to remove dust entrained between pins and wiring and to prevent re-distribution of asbestos.
• Cleaning procedures for distribution frames were provided previously and are included in Section 11.0 of this report.
• All facility and network support elements including cable racks, light fixtures, storage cabinets, and framework in this space should be HEPA vacuum cleaned.
• In addition to Telcordia procedures for cleaning contaminated telecommunications distributing frames, all Verizon protocols for cleaning asbestos containing dust should be understood and followed.

 

Power Room

• Light accumulation of dust with trace asbestos was      observed within the 9^th floor power room, likely attributable      from emergency restoration activities and cross contamination from heavily      contaminated areas on the same floor.
• All accessible power equipment surfaces, cable racks,      and facility surfaces should be HEPA vacuum cleaned by a person      experienced in working on DC power plant, following all safety precautions      for live bus bar, rectifiers, BDFBs, etc.
• The flooded cell batteries should be wiped with a      tack cloth or non-shedding cloth lightly dampened with distilled or      deionized water to remove any remnant dust and debris.
• Cleaning procedures for power rooms are included in      Section 11.0 of this report.

 

Collocation, Voice Mail, Storage and Mechanical Room

• These spaces sustained no direct damage but were contaminated with light amounts of asbestos containing dust. These areas should be HEPA vacuum cleaned to remove particulate contamination from all accessible facility and external equipment surfaces. These surfaces include light fixtures, cable racks, framework, cages, demarcation points, floor, as well as stored supplies such as cable reels and boxes.

 

Observations

 

Nortel DMS-100 Switch

The 9^th Floor DMS-100 switch sustained severe contamination by asbestos containing particulate, concrete dust and abrasive fragments, and glass.  The contamination entered through blown in windows on the east and south walls of the switch room, and it became entrained within the DMS-100 electronics.  Furthermore, one signal cable rack was somewhat deformed, and cabling penetrations sustained abrasion and nicks from compacted glass fragments.   Photographs 6-1 through 6-6 below present the condition of the DMS-100 switch space as of September 12, 2001.   The broken windows and loss of environmental control resulted in high temperature and humidity concerns within the space.  The temperature concerns are increased due to the clogged frame filters on fan-cooled frames, severely limiting the cooling capacity. Additional internal heat dissipation difficulty was caused by the thermal blanketing of the electronics with dust. The thermal concern is described in detail in Section 10.

                         

Photograph      6-1. Blown out      windows and heavy dust and debris throughout the DMS-100 switch room.

                            

Photograph      6-2.  Severe dust contamination covering all      DMS-100 surfaces severely loaded frame filters.

    

 

 

 

 

 

 

 

 

 

 

 

 

 

 

            

 

 

 

 

           

 

 

 

 

                       

Photograph      6-3.      Deformation of cable rack and likely stress on signal cables serving the      DMS-100.

                              

Photograph      6-6. Additional      debris within the DMS-100 switch room.

                            

Photograph      6-5.  Additional blown out window in DMS-100      room permitting ingress of particulate and exposure to untemepered      air.

                            

Photograph 6-4.  Blown      out windows on south wall and glass and debris that may have damaged      signal cable at floor penetration.

    

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

As shown by the above photographs, the physical damages to the DMS-100 space resulted in severe particulate contamination throughout the switch room. Areas of larger debris, glass, and concrete fragments also entered the space and impacted the equipment closest to the east and south walls as well as collected on and between cable risers.  Also, the photographs illustrate concerns with possible cabling damages.

Photograph 6-7. Very heavy layer of asbestos containing   particulate contamination on a shelf within a DMS line card frame.

 

After the initial bulk cleanup was completed, Telcordia re-visited the site for a more thorough evaluation of the DMS-100.  While much of the contaminant had been cleaned from facility surfaces and accessible portions of the switch, extensive contamination remained within the switch, including circuit packs. Contamination also was noted between signal and power cables on the racks and under the raised floor.  Photographs 6-7 through 6-9 illustrate the contamination.  These areas are within the switch framework and cannot are removed without partial disassembly of frames as well as complete removal of circuit packs.

Photograph 6-8.    Heavy particulate contamination containing asbestos fibers behind the   front grating of a DMS switch frame requiring disassembly to adequately   clean.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 6-9.    Particulate adhered between IC component leads on BIC card in a line   card drawer.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Switch frames in line-ups at greater distance from the damaged east and south walls had somewhat lesser contamination than the frames immediately in the path of the open broken windows, but all frames had notable internal particulate contamination.  The presence of asbestos fiber in the particulate makes it highly recommended that all of the contaminant be abated.

 

While the frame filters were severely loaded “clogged” with particulate, the frames remained operational via battery after loss of commercial power and subsequent loss of environmental control. As such the switch modules were subjected to room temperatures exceeding 100 degrees F. The chart recorder within the DMS-100 indicated that the temperature within the space had reached a maximum of 118 degrees F.   Although this is just below the NEBS design criteria for short-term (96 hour) operation of 122 degrees F, it is a major concern that was exacerbated by the reduced airflow due to clogged frame filters and by thermal stress on electronic components resulting form reduced heat dissipation from thermal blanketing by dust.

 

 

Photograph 6-10. DMS-100 Room chart   recorder showing a temperature of 118 degrees F was recorded in the space.

It is likely that a significant number of circuit packs will become problematic as telephone traffic increases. Section 10.0 of this report provides more detail on thermal concerns with the equipment.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In addition to the direct damage and contamination, the location of the DMS-100 will present difficulties during the building structural repair.  It is anticipated that the building repair will require removal and replacement of main structural columns along the east wall and extensive building facade repair. The location of the DMS within only a few feet from the damaged exterior walls will make construction/renovation work difficult or impossible without disrupting the network services. Accessibility to the DMS-100 will be impeded, and the work activities may further damage the switch or switch support structures.  The significant work that is required on the east and south walls immediately around the switch will make it impossible to provide a controlled and accessible environment for the switch, and for Verizon craft to adequately maintain the DMS-100 for routine and emergency services. Significant building work on floors immediately above and below the switch room also raises concerns and communication cables routed to the switch will be vulnerable during facility repairs.  Other logistics and network engineering problems will likely arise with re-cabling of interconnected equipment as well as cleaning and repairing contaminated cable risers and trays.

 

Analyses of five (5) 9^th floor particulate samples (Table 8-1) for asbestos via PLM by a third party laboratory detected asbestos at a level greater than one percent in one sample (9^th Floor Frame Area, 9/13; less than one percent in one sample (9^th Floor DMS-100, 10/15), and did not detect asbestos in three additional samples.  Electron microscope observations by Telcordia identified asbestos fibers within all remaining samples collected on the 9^th floor.  Section 8.0 of this report describes these findings in greater detail and includes electron microscope photographs of the asbestos.   The particulate is light and easily becomes airborne when disturbed.  Telcordia recommends that all internal and external DMS-100 frames, associated cabling, framework and hardware located on Floor 9, including under raised floor be cleaned. See DMS cleaning procedures in Section 11.0. This cleaning will help to minimize personnel exposure to particulate as well as reduce the potential for redistribution and cross-contamination of network equipment and facility. Alternately, Verizon may opt to provide asbestos level compartmentation of the DMS space and when the switch is removed, encapsulate all frames using two layers of plastic sheeting (6 mil minimum thickness) to prevent cross contamination without performing full detailed cleaning.

 

Analyses of surface residue via ion chromatography revealed no elevated levels of ionic contaminants (See Table 8-7 in Section 8.0), therefore, ionic contamination of 9^th floor network equipment is not a concern.

 

The detailed internal and external cleaning of DMS-100 is expected to be labor intensive, requiring removal of all circuit packs, partial frame disassembly, and the potential for extended loss of service within frames when they are being cleaned. This level of cleaning and restoration is required to alleviate asbestos concerns, and to restore the switch to a pre-event condition.  The restoration must also take into account building renovation and construction activities in close proximity to the switch that will limit accessibility to the switch and pose a significant risk for service interruptions and physical damages.

 

Verizon must also consider all warranty and service impacting issues, the unknown cumulative impacts from the event on long-term reliability as well as the projected costs to accomplish the restoration.

 

Cosmic Frame

The cosmic frame is located in an adjacent room to the DMS-100 switch room.  The frame also sustained heavy particulate contamination and entrainment.  The particulate also contained larger abrasive fragment of cement based materials.  The Telcordia chemical analyses and independent York State PLM testing showed the presence of significant asbestos fibers (2.3%) within the particulate. Cable racks, framework, and facility surfaces also sustained serious contamination with asbestos containing particulate.

 

The initial bulk cleaning was effective in removing the surface dust from contacts, pinfields, and wiring. Harder to reach spaces, between wiring and within the frame, still exhibited concern contamination with asbestos containing dust.  In addition cable racks and support structures were still heavily loaded with particulate.  Photographs 6-11 through 6-13 illustrate the remaining frame contamination concerns.

 

Photograph 6-11.  Asbestos dust remaining on horizontal   surfaces within the cosmic frame.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 6-12.  Asbestos dust remains on   frame pin terminations.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 6-13. Asbestos dust on   horizontal block pins and horizontal surfaces following initial cleaning.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

No indication of corrosive damage or contact with water was observed on the frame, and there was no observed physical damage.  Also, the degree of contamination is not currently service affecting but will require detailed abatement to return to a pre-event condition.

 

Telcordia recommends full HEPA vacuum cleaning of the frame and the frame room in accordance with both Telcordia cleaning procedures for central office environments along with full asbestos abatement protocols. The Telcordia procedures for cleaning the distributing frame are provided in Section 11.0 of this report.

 

Power Room

The power room on the ninth floor serves the DMS-100 switch and is located in the north side of the building.  No physical damages were observed.  Likewise, no water damage or high relative humidity exposure concerns were noted within the power room. Examination of exposed copper bus serving the batteries and BDFBs as well as internal rectifier electronics exhibited no signs of corrosive damage or excessive particulate ingress. Minor dust accumulations were present on

 

horizontal surfaces of the equipment consisting of typical dusts along with some building materials.  Some of this ingress of building related dusts is likely due to cross contamination during emergency power work or facility cleaning of adjacent spaces, etc.

 

While the power room space is relatively clean, the cross-contamination issue warrants general HEPA vacuum cleaning to restore the space to a pre-event condition and as a precaution due to the presence of trace asbestos dust in the power room as well as more significant asbestos in other areas of the 9^th floor, e.g., the DMS-100 and Cosmic frame spaces. Our initial testing of particulate from this space in September did not reveal asbestos, however, samples evaluated in October began to show trace asbestos fibers. The HEPA vacuum cleaning should be performed for all accessible horizontal and vertical surfaces in the space from the cable racks and light fixtures down. All cleaning must be performed by persons knowledgeable in DC power room safety precautions including the use of insulating rubber mats on exposed bus, insulating vacuum hoses, and proper grounding requirements. The flooded cell batteries should be wiped with a tack cloth or a non-shedding cloth lightly dampened with distilled or deionized water to remove any remnant dust and debris. Cleaning procedures are included in section 11.0 of this report.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 7.0

FLOORS 1, 3, 6, and 8

Assessment of Distribution Frames and Power Rooms

 

 

 

 

 

 

 

Overview: Floors 1, 3, 6, and 8.

The remaining network equipment floors in 140 West Street are presented in this section since each floor contains a limited amount of network elements.  The Main Distributing Frame (MDF) is located on the first floor along Vesey Street and is adjacent and parallel with the building lobby.  Floors 3 and 6 contain network equipment power rooms, and Floor 8 has an additional Distribution Frame.  The 1^st floor MDF space and the 8^th floor frame were directly impacted by the collapse with significant particulate and debris contamination, while the two power rooms were not damaged but do exhibit contamination concerns.

 

Summary of Conclusions

1^st Floor MDF & Fiber Cross Connects

• The first floor lobby was open to the outside and used for firefighting resulting in heavy dust and debris contamination.
• The MDF space sustained heavy particulate and debris contamination associated from broken windows, structural damage, and the collapse of a non-bearing sheetrock wall that runs parallel with frame section F45.
• The MDF and adjacent fiber cross connect frame exhibited minor corrosion from water contact near the Washington Street & Vesey Street corner of the room.
• The particulate in the lobby was found to contain asbestos fiber via PLM analysis and Telcordia ESEM analysis identified asbestos fibers in both the lobby and MDF areas.
• Frame section F45 and fiber cross-connect frames located next to F45 on the Vesey Street side, should be replaced due to corrosion, physical damage and severe contamination.
• Also, frame section 45 and the fiber cross-connects will likely require replacement to permit facility structural repairs.
• Remaining sections of the 1^st floor MDF should be fully HEPA vacuum cleaned to remove all dust and debris and to fully abate the asbestos containing particulate contamination. Cleaning procedures are provided in Section 11.0.

 

3^rd Floor and 6^th Floor Power Rooms

• The power rooms were intact with no broken windows, no visible structural impact, and were initially free of significant contamination as assessed on September 12 and 13.
• No physical damages to batteries occurred and there was no evidence of electrolyte leakage or battery “walking”.  However, the deep discharge to 1.35 volts of the batteries associated with the loss of primary AC power, resulted in cell reversal, requiring select cell replacement.
• Subsequent visits to the power rooms in mid and late October, revealed notable contamination had occurred on equipment and facility surfaces likely due to building renovation activities and cross contamination from other areas of the building. The particulate tested positive for traces of asbestos fibers by Telcordia ESEM analysis.
• All external facility and power equipment surfaces in both power rooms should be thoroughly HEPA vacuum cleaned. Batteries and stands should be wiped clean with a non-shedding lint cloth moistened with deionized or distilled water. Cleaning procedures, included in Section11.0, should be performed by persons familiar with precautions for working on power equipment and exposed bus.
• Once cleaning is complete, positive pressurization and controlled access should be implemented to prevent additional contamination.

8^th Floor Frame

• The distribution frame area sustained heavy contamination from asbestos containing particulate due to external wall damage near the corner of Washington and Vesey Streets.
• Verizon should determine if the extensive building structural renovation near the frame will impede access to the frame and require moving customer line terminations to another frame or to relocate frame terminations to areas of the existing frame away from the concern area.
• Once building renovation logistics concerns are addressed and access to the frame is ensured, the frame can be retained and used after full HEPA vacuum cleaning and asbestos abatement as outlined in procedures provided in Section 11.0.

Photograph 7-1.  Damaged sheetrock wall along the MDF, along   with debris and contamination.

 

Observations

Floor 1. MDF and Lobby

The Main Distributing frame (MDF) is located in a dedicated room adjacent to the building lobby that runs parallel with the Vesey Street side of the building.  The building lobby area and MDF space had been utilized by firefighting and emergency personnel immediately after the World Trade Center collapse and were significantly contaminated with dust and debris.  The dust collected from the lobby tested positive for asbestos fibers using the PLM method and dust removed from the MDF and lobby tested positive for asbestos via Telcordia ESEM analysis. Windows on the west side of the MDF room were broken resulting in ingress of debris and particulate. A sheetrock (non load bearing) wall along the frame was buckled and partially collapsed, resulting in additional dust and debris contamination of the frame section F45.  Areas of water damage were also present on a portion of the frame along the Vesey Street wall, east of F45.  A fiber cross connect frame adjacent to the east end of the frame also sustained water damage and severe particulate ingress.

 

The majority of the frame can be retained and used after full HEPA vacuum cleaning is completed as outlined in Section 11.0 of this report.  However accessibility of the frame near the southeast corner of the building may be compromised during the time of building structural renovations.

 

Floors 3 and 6. Power Rooms

Two power rooms are located on the sixth floor. One is located along the north wall of the building and the other is totally internal to the building behind the elevator bank on the West Street side. Neither space sustained physical damages in the attack and no damage from firefighting or emergency procedures were noted.   The equipment within the power rooms included Lorain rectifiers, flooded cell battery strings, power distribution bays, and associated copper distribution buss and cabling. Initially, only light particulate contamination was observed in these spaces and it could not be differentiated from typical building dusts. However, subsequent sampling in October indicated the presence of asbestos fibers co-mingled with the heavier “white” particulate.

 

We attribute this to cross contamination from common areas of the sixth floor that suffered heavy physical damage to exterior walls with particulate ingress from the WTC as well as the generation of renovation dust during emergency restoration.  Particulate was evident on all horizontal surfaces including the top of batteries, rectifiers, and on all horizontal cable and cable racks.  Photographs 7-2 and 7-3 illustrate the dust levels on batteries and a rectifier shelf.

Photograph 7-2.  Heavy dust contamination on the top of a   flooded cell in a battery string in the 6th floor power room.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Photograph 7-3.  Significant dust contamination on a   horizontal shelf of a Lorain rectifier frame.

 

Likewise, the power room located on the third floor also did not suffer physical damage and was initially free of significant contamination as sampled in September. However, subsequent sampling in October indicated the presence of trace asbestos fibers co-mingled with the heavier “white” particulate.

 

Telcordia recommends that power rooms on floors 3 and 6, be cleaned per procedures as outlined in section 11.0.  The cleaning must be conducted by persons familiar with power plant systems following all safety procedures for working on and around exposed buss, rectifiers and batteries.

 

Floor 8. Distributing Frame

The 8^th floor sustained significant structural damage to the external wall facing Washington Street, resulting in heavy ingress of debris and particulate.  The two tier distribution frame, that runs parallel with Vesey Street, was exposed to the damages and was severely contaminated with particulate that was found to contain asbestos fibers.

 

Due to the proximity of the frame to the side of the building that will require extensive structural renovations, it is likely that access to the frame will be impeded.  If logistics and accessibility issues are resolved for the frame, it can be restored by cleaning as outlined in Section 11.0.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 8.0

Chemical Analyses

 

 

 

 

 

 

 

 

 

Overview

Telcordia Technologies performed on-site qualitative and off-site quantitative chemical analyses in order to evaluate the nature and potential impacts of residues deposited onto equipment and facility surfaces as the result of the events of September 11.  Quantitative analyses to determine the elemental and ionic composition of particulate residues and gas phase products of combustion were performed in our laboratory in Red Bank, New Jersey.  Quantitative analyses of particulate residues for asbestos content were performed by EMSL (307 West 38^th St., New York, New York), a contracted laboratory, while qualitative assessment for the presence of asbestos fibers in particulate were performed at Telcordia. The range of tests performed as part of our evaluation included surface pH measurements, copper mirrors corrosion, environmental scanning electron microscopy (ESEM), Polarized Light Microscopy (PLM), ion chromatography (IC), and surface insulation resistance testing. The results of these analyses are presented in the following sections.

 

Summary of Conclusions

Asbestos Analyses

• 10 of 26 samples tested exhibited levels of chrysotile asbestos detectable by Polarized Light Microscopy (PLM).
• 4 of those 10 samples exhibited greater than one percent asbestos composition, the level defined for building materials by the EPA as asbestos-containing.
• Utilizing an Environmental Scanning Electron Microscope (ESEM), Telcordia initially detected asbestos fibers in 19 of the 22 samples, including 8 bulk samples for which PLM analyses did not indicate asbestos.  See Table 8-1.
• A total of 88 samples analyzed via ESEM at Telcordia exhibited various degrees of asbestos fibers.  Transmission Electron Microscopy (TEM) analyses, not conducted as part of this investigation, would more precisely quantify the level of asbestos in particulates where PLM detected asbestos at levels below one percent.
• Comparisons of certain samples in Table 8-1 indicate that surfaces that were once void of asbestos were later contaminated.  Testing of particulate from the 3^rd and 6^th floor power rooms and the 7^th floor DMS in the middle of September did not reveal the presence of asbestos.  Subsequent sampling of these areas in the middle of October revealed the presence of trace asbestos fibers on various external equipment surfaces in these areas.  Contamination of these surfaces is occurring due to ongoing equipment repair and replacement activities, migration or entrainment of particulate matter into the building from exterior sources, and/or cross-contamination by particulates generated by HVAC systems, and other demolition, construction, and ongoing equipment management activities throughout various areas of the facility.
• Telcordia recommends that, where directed in Sections 3.0 through 7.0 of this report, all internal and external network equipment, cabling, framework, hardware, and facility surfaces be cleaned.  Cleaning procedures for the various systems specified are included as Section 11.0.  Cleaning will help to minimize personnel exposure to particulate as well as reduce the potential for redistribution and cross-contamination of network equipment and facility.  Alternatively, Verizon may opt to provide asbestos level compartmentation in the specified equipment spaces, then when the equipment is removed, encapsulate all frames using two layers of plastic sheeting (6 mil minimum thickness) to prevent cross contamination without performing detailed cleaning.
• Should workers disturb asbestos-containing particulates remaining on uncleaned equipment surfaces and facility surfaces, local exposures to individuals may be of concern, even if air monitoring in the equipment spaces does not indicate elevated levels of airborne asbestos.  It is our professional opinion that Verizon continue remediation of affected areas of the office as the most prudent course of action.

 

Please note that Telcordia Technologies’ evaluation of the presence or absence of asbestos is not designed to take the place of environmental or occupational evaluations that may be required under applicable federal, state, and local laws and regulations.  Telcordia recommends that this information be shared with Verizon’s Environmental, Health, and Safety organization for review, and if necessary, further investigation.

 

Elemental Analyses

• Spectra of samples of particulates collected from floors 1, 2, 3, 4, 6, 7, 8, and 9 revealed that certain elements were typical of particulates throughout the evaluated portions of the building.  These included magnesium, calcium, silicon, sulfur, aluminum, sodium, chlorine, potassium, and iron.  These elements indicate the ingress into the Verizon building by large amounts of particulate generated by the collapse of the World Trade Center buildings.
• Several systems on the 1^st, 2^nd and 4^th floors of the building sustained water damages that resulted in corrosion.  Such systems included D4 Channel Banks, SLC Series 5, Telecom Solutions Clocks, Distributing Frames, Nortel DMS Urban Switch, SMAS, Fujitsu FLM and NEC muxes, fuse shelves, Lucent DACS, DSC CS1, etc., discussed in Sections 3.0 and 4.0.  Analyses of corrosion residues indicated the presence of   products of chemical, galvanic, and electrolytic corrosion.  These corrosion damages are irreparable.

 

Ion Chromatography

• The primary ionic constituents detected in the 195 collected residue samples were chloride, nitrate, and sulfate. 
• None of the results for chloride or nitrate exceeded Telcordia criteria of 50 mg/in² for reliable operation of network equipment.
• Sulfate data exceeded our criteria on localized areas of the 2^nd, 4^th, and 6^th floors.  Rates of exceedance were 14% and 9% for the 2nd and 4th floors, respectively, and one of three samples from the 6^th floor.  However, no consistent spatial relationship exists between locations where excess sulfate were detected.  Telcordia believes that these represent sulfate in particulate captured on the sampling media rather than gas phase products of combustion deposited directly onto equipment surfaces from smoke ingress, and therefore, is not of further concern.  No remedial action is required to address ionic contamination following the September 11 attacks.

 

Other Analyses

• pH measurements of residues throughout the floors evaluated indicated that the pH values of equipment and facility contaminants exhibited highly alkaline values of 12 to 14. These residues become corrosive to metallics when damp or wet.  Such corrosion was observed on water-damaged equipment, and is a future risk for remaining contaminated, undamaged equipment.
• Copper Mirror Corrosion: Particulate samples dissolved in deionized water and placed on a copper mirror indicated that particulates from this event were moderately corrosive.
• Insulation Resistance Test:  Review of the data indicates that the surface resistance of the particulate collected from the office dropped about two orders of magnitude at relative humidity levels between 70 and 98 percent.  The data indicate that the particulates tested are not hygroscopic, or of concerns in environments of high relative humidity.

 

Chemical Evaluation

 

Asbestos Analyses

Analyses were conducted on 26 samples of bulk particulates, best described as a as a uniform brown/gray fluffy fibrous material, collected from selected locations within floors 1, 3, 4, 6, 7, 8, and 9 of the office.  These analyses were performed to assess the presence or absence of asbestos materials in the particulates deposited on equipment and facility surfaces as a result of the September 11 events.  Telcordia submitted samples to a laboratory accredited by the State of New York for the bulk analyses of asbestos-containing materials (EMSL, Inc., 307 West 38^th Street, New York City, 212-290-0051, NVLAP#101048-9, NY ELAP #11506).  Telcordia also qualitatively analyzed multiple samples via ESEM in our Red Bank, New Jersey laboratory. Please note that Telcordia Technologies’ evaluation for the presence or absence of asbestos is not designed to take the place of environmental or occupational evaluations that may be required under applicable federal, state, and local laws and regulations.  However, efforts to evaluate the presence of asbestos in the areas we investigated have produced data that is instructive regarding the nature of the deposited materials. Telcordia recommends that this information be shared with Verizon’s Environmental, Health, and Safety organization for review, and if necessary, further investigation.

 

Different methods were utilized in the analysis conducted by EMSL and Telcordia.  EMSL analyzed the samples for the presence or absence of asbestos fibers using Polarized Light Microscopy (PLM) according to New York State ELAP Method 198.1.  This is a point counting method focused on identifying and quantifying any fibers with an aspect ratio of 5:1 or greater and lengths exceeding 5 microns.  However, our own ESEM observations, reports from personnel at other testing laboratories, and articles in the popular press indicate that asbestos fibers dispersed by the events of September 11^th are present in sizes smaller than 2 microns. These fibers are below the standard PLM test method sensitivity and, therefore, are not counted under PLM methods.  The analytical results from both EMSL and Telcordia efforts are presented in Table 8-1 below.  Images 1 through 33, which follow at the end of this section, illustrate electron microscope images recorded at Telcordia that depict the presence of asbestos fibers, which are denoted by red arrows.  At least 84 of 88 samples analyzed at Telcordia exhibited asbestos fibers.  Other fibrous materials visible in the images are not characteristic of asbestos, i.e., they are greater than 3 microns diameter or have morphological characteristics not associated with asbestos fibers.  Instead, we attribute the larger or uncharacteristic fibers to other common materials, such as fabric, filter media, insulation, pulp, mold spores, etc. 

 

 

                    

TABLE 8-1    BULK PARTICULATE ASBESTOS ANALYSIS
NO.	LOCATION/DESCRIPTION	BULK ANALYSIS VIA PLM	% ASBESTOS FIBER VIA PLM	ASBESTOS FIBERS VIA ESEM
1	WEST STREET   FRONT FACADE, 9/13	POSITIVE	2.50%	POSITIVE
2	FRONT LOBBY,   9/13	POSITIVE	<1.00%	POSITIVE
3	9TH FLOOR POWER ROOM, 9/13	NEGATIVE	—	NOT DETECTED
4	9TH FLOOR POWER ROOM, 10/18	NEGATIVE	—	NOT DETECTED
5	9TH FLOOR DMS-100, 9/13	NEGATIVE	—	POSITIVE
6	9TH FLOOR COSMIC   FRAME, 9/13	POSITIVE	2.30%	POSITIVE
7	9TH FLOOR DMS-100,   10/18	POSITIVE	<1.00%	POSITIVE
8	8TH FLOOR OFFICE   SPACE, 9/13	POSITIVE	<1.00%	NOT TESTED
9	7TH FLOOR 5ESS   SPACE, 9/13	POSITIVE	2.40%	POSITIVE
10	7TH FLOOR 5ESS, DS1, 10/18	NEGATIVE	—	POSITIVE
11	7TH FLOOR 5ESS, DS1, 10/18	NEGATIVE	—	POSITIVE
12	7TH FLOOR DMS-100, 9/13	NEGATIVE	—	NOT DETECTED
13	7TH FLOOR DMS-100, 10/16	NOT TESTED	—	NOT DETECTED
14	7TH FLOOR   COLLOCATION ROOM, 9/13	POSITIVE	1.80%	POSITIVE
15	7TH FLOOR   MECHANICAL ROOM, 9/13	POSITIVE	<1.00%	POSITIVE
16	6TH FLOOR POWER ROOM, 9/14	NOT TESTED	—	NOT DETECTED
17	6TH FLOOR POWER ROOM, 10/18	NOT TESTED	—	POSITIVE
18	4TH FLOOR TRANSPORT ROOM,   9/14	NEGATIVE	NOT DETECTED	POSITIVE
19	4TH FLOOR TELLABS, RR 4413,   10/18	NEGATIVE	NOT DETECTED	POSITIVE
20	4TH FLOOR DSC   SHELF, RR 4019, 10/18	POSITIVE	<1.00%	POSITIVE
21	3RD FLOOR POWER ROOM, 9/13	NOT TESTED	—	NOT DETECTED
22	3RD FLOOR POWER ROOM, 10/18	NOT TESTED	—	POSITIVE
23	2ND FLOOR D4 SPACE, RR 2013,   9/14	NEGATIVE	NOT DETECTED	POSITIVE
24	2ND FLOOR D4 FRAME,   RR 2008, 10/18	POSITIVE	<1.00%	POSITIVE
25	2ND FLOOR D4 SPACE, RR 2228,   10/18	NEGATIVE	NOT DETECTED	POSITIVE
26	2ND FLOOR D4 SPACE, RR 2406,   10/18	NEGATIVE	NOT DETECTED	POSITIVE
27	1ST FLOOR MDF, 9/14	NEGATIVE	NOT DETECTED	NOT TESTED

 

Review of the data in Table 8-1 above reveals that:

1. 10 of the 26 samples analyzed by PLM exhibited detectable levels of asbestos.
2. 4 of those 10 samples exhibited greater than one percent asbestos composition, the level defined for building materials by the EPA as asbestos-containing. (Asbestos-containing building materials trigger proactive or remedial actions, under EPA and other regulatory codes and standard real estate management practice.)
3. Utilizing ESEM, we observed asbestos fibers in 19 of the 26 samples discussed above, including 8 bulk samples for which PLM analyses did not indicate asbestos content.  Analyses of samples containing asbestos fibers exhibited elevated levels of magnesium silicate.  Additionally, as previously mentioned, asbestos fibers were also observed in at least 84 of the 88 tape samples we analyzed from various portions of the building.  Fiber lengths were as short as 2 microns. This finding lends credence to discussions with testing labs and popular press reports indicating that, as a result of the immense forces involved in the World Trade Center collapses, asbestos materials demolished in the events generated debris that included a proportion of very small fibers (below 5 microns length).  As a result, asbestos fibers are present in the World Trade Center area that are smaller than those detected by conventional PLM test methods, and are not being addressed due to underreporting by standard analytical methods. 
4. Asbestos is present in particulate on equipment and facility surfaces in spaces where Verizon is currently performing air sampling for airborne asbestos. Verizon is assumed to be finding concentrations to be below levels that warrant personal protective equipment.  Should workers disturb asbestos-containing particulates remaining on internal equipment surfaces and uncleaned facility surfaces, local exposures to specific individuals may be high where they are working, even if air monitoring in the equipment spaces does not indicate elevated levels of airborne asbestos within the room.
5. Comparisons of certain samples in Table 8-1 (Sample 5 with 7, 16 with 17, 21 with 22) indicate that surfaces that were void of asbestos in September were later contaminated with asbestos fibers, as indicated by October sampling.  Contamination of these surfaces is occurring due to ongoing equipment repair and replacement activities, migration or entrainment of particulate matter into the building from exterior sources, and/or cross-contamination by HVAC or particulates generated by other demolition, construction, and equipment management activities ongoing throughout various areas of the office.

 

Overall, the data indicates that asbestos is present on equipment and facility surfaces in most of the areas sampled.  Table 8-1 above shows that asbestos was detected at levels above one percent in the 1^st Floor Lobby, 7th floor 5ESS, 7^th floor collocation room, 9^th floor cosmic frame area, and the front façade just outside the main entrance. These levels typically require abatement as described for asbestos-containing building materials by the EPA, NYSDEC, NYCDEP and any other applicable regulatory agencies. Trace levels of asbestos below 1% were detected at various locations within the 1^st, 2^nd, 3^rd, 4^th, 6^th, 7^th, 8^th, and 9^th floors.  Recontamination of cleaned surfaces is ongoing via cross-contamination mechanisms observed on-site.  Disturbance of asbestos-containing particulates on facility surfaces and external and internal equipment surfaces, including removal of circuit packs for service, could result in high, short-term, local exposures to workers, and result in cross-contamination of facility and equipment surfaces.

 

Currently, there is no current consensus on the health effects of asbestos fibers below five microns length.  Conflicting information has been observed in the popular press regarding this subject, and published statements by noted health institutions have raised concerns regarding the health affects of small asbestos fibers.  We are not aware of any regulation specifying a level at which asbestos-containing surface particulate must be remediated, however, given all of the factors above, it is our professional opinion that Verizon strongly consider remediation of settled particulate in the affected areas of the office as the most prudent course of action.  Telcordia continues to recommend, consistent with our previous reports for this event, that all particulates on facility and equipment surfaces be treated as asbestos-containing, and that asbestos-containing particulates be cleaned as noted in our schematics and recommendations in Section 2.0.  Cleaning procedures are detailed in Section 11.0.

Image   1. Asbestos   fibers in particulate removed from a 2nd fl. D4 2FXO Circuit Pack in RR2111.03

Image   2. Asbestos   fibers in particulate collected from a 2nd fl. test points in   RR2201.17 80’ from east wall.

 

Image   3. Asbestos   fibers in particulate removed from a 2nd fl. SMAS relay in RR 2300, 120′ from east   wall.

Image   4. Asbestos   fibers in particulate collected from a 2nd fl. Cross-Connect Frame, 140′ from east wall.

 

Image 5. Asbestos fibers in particulate removed from   a 2nd fl. SMAS   shelf in RR  2101.14 (mixed asbestos   and conventional fibers).

Image   6. Asbestos   fibers in particulate collected from a 2nd fl. Lightguide fiber distribution frame RR   236.14.

 

Image   7. Asbestos   fibers in particulate removed from a 3rd fl. ac distribution panel in PDB 3018.01.

Image   8. Asbestos   fibers in particulate collected from the 4th fl. DACS IV, RR 4015.00.

 

Image   9. Asbestos   fiber in particulate removed from a 4th fl. Cisco 7500 Router in RR 4903.01.

Image   10. Asbestos   fibers in particulate collected from 4th fl. Fujitsu FLM 600 in RR 4804.11.

Image   11. Asbestos   fiber in particulate removed from 4th fl. Tellabs Titan 5500 equipment in RR   4416.03.

Image   12. Asbestos   fiber in particulate collected from the 4th fl. DSC K33 MUX in RR 4103.04.

 

 

 

 

 

Image   13. Asbestos   fiber in dust removed from the 4th fl. Telecom DCD 523 in RR 4607.00.

Image   14. Asbestos   fibers in dust collected from 4th fl. Alcatel DSLAM equipment in RR4513.04.

 

Image   16. Asbestos   fibers in dust collected from the 6th fl. power room Lorain Flotrol Rectifier G-6 in   PBD0602.06.

Image   15. Asbestos   fibers in dust removed from 4th fl. Alcatel 1631 MUX in RR    4302.11.

Image   17. No   asbestos was seen in dust removed from the 7th fl. DMS DTEI 15 (frame H22) in September.

Image   18. However,   asbestos fiber was present in October in dust collected from 7th   fl. DMS LGEI 001 filter   grating (frame J10) .

 

 

 

 

Image   19. Asbestos   fibers in dust removed from a 7th floor file cabinet surface near   windows.

Image   20. Multiple   asbestos fibers in dust collected from 7th fl. 5ESS DS0 SM101 LTP0.

Image   21. Multiple   asbestos fibers in dust removed from the 7th fl. 5ESS DS0 Cable Rack above SM101.

Image   22. Asbestos   fibers in dust collected from the. 7th fl. 5ESS DS0 SM76 LTP2.

Image   23. Asbestos   fibers in dust removed from 7th fl. 5ESS DS1 DSC 03.

Image   24. Asbestos   fibers in dust collected from 7th fl. 5ESS DS1 SM002 LTP2 LU22.

 

 

 

Image 25. Multiple asbestos fibers in dust removed   from 7th fl.   5ESS DS1 SM009 LTP0.

Image 26. Asbestos fiber in dust collected from 9th   fl. DMS LGE 001 (frame J02).

 

Image   27.   Multiple asbestos fibers in dust removed from 9th fl. windowsill   adjacent to DMS DTE 024 (frame T01).

Image   28. Multiple   asbestos fibers in dust collected from 9th  fl. DMS LCE 028 (frame B02).

 

 

Image   29.   Asbestos fiber in dust removed from the top of a battery string in the 9th   fl. Power Room.

Image   30. Asbestos   fiber in dust collected from the 9th fl. Cosmic Frame, shelf 19-2.

 

Image   31.   Multiple asbestos fibers in dust removed from the 9th fl.   Mechanical Room.

Image   32.   Multiple asbestos fibers in dust collected from the 9th Floor   Nextlink Cross-Connect in RR 9072A.03.

 

Image   33.   Multiple asbestos fibers in dust removed from the 9th fl. Voice   Systems equipment.

 

Important Notes Regarding Asbestos Analyses

The analytical techniques utilized in evaluating the particulate included Polarized Light Microscopy (PLM) and Environmental Scanning Electron Microscopy (ESEM).  PLM methods typically quantify fibers over five microns in length by their aspect ratio (> 5:1), but do not differentiate asbestos from non-asbestos fibers.  As such, the actual percentage of asbestos in the samples analyzed may be either lower than reported due to incorporation of non-asbestos fibers into the results, or higher due to the inability to account for fibers smaller than five microns.  Despite the potential for these inaccuracies, PLM analysis is widely used because it can be performed rapidly at relatively low cost.

 

The ESEM analyses conducted by Telcordia allows for the visual observation of fibers below five microns in length, and when equipped with EDX systems such as ours, enables the differentiation of asbestos and non-asbestos fibers.  As previously noted, our observations indicate the presence of asbestos fibers smaller than those reported under PLM methods in the bulk samples, and identified asbestos fibers in the vast majority of the tape samples collected.  However, to our knowledge, there is no standard asbestos counting method associated with ESEM analyses, therefore, ESEM data cannot be utilized for the quantitation of asbestos.

A third technology, Transmission Electron Microscopy (TEM), maintains sensitivities comparable to ESEM, and utilizes counting methods that do quantify asbestos in bulk samples.  When bulk or air samples analyzed by PLM indicate high results, TEM is typically employed in order to differentiate and quantify actual asbestos fibers from non-asbestos fibers.  TEM is also utilized to evaluate the presence or absence of smaller fibers not counted by PLM.  As such, TEM data is the best and most reliable method for determining the level of asbestos content f6r this type of particulate.  TEM analyses are, however, considerably more expensive than PLM analyses.  Telcordia recommends that Verizon utilize TEM asbestos analyses on samples collected by Verizon or its contractors.

 

Elemental Analysis

 

Telcordia collected 147 samples of particulate from facility and equipment surfaces.  Initially, 88 representative samples were analyzed in our laboratory in Red Bank, New Jersey.  Samples were collected from equipment and facility surfaces by using transparent adhesive tape with a known reference spectrum, or in bulk form.  An Environmental Scanning Electron Microscope with Energy Dispersive Spectrometer (ESEM/EDS) was used to examine the characteristics of the particulate and to obtain spectra of the relative intensities of all elements constituting the sample.  Common sources for the elements detected are described in Table 8-2 below.

        

TABLE 8-2    TYPICAL CENTRAL OFFICE DUST SOURCES
ELEMENT	SOURCES
Aluminum	Soils, building materials, hardware installation
Bromine	Fire retardant plastics
Calcium	Road salt, sea salts, soils, cementitious materials
Chlorine	Plastic degradation, solvents, water treatment, refrigerants, sea   salts
Iron	Ferrous metals, soils
Magnesium	Humidification, concrete, soils, building materials, asbestos
Phosphorous	Detergent, dry chemical fire extinguisher
Potassium	Humidification, fertilizer, soils, paints, sea salts, cigarette smoke
Nickel	Metal plating, stainless steel alloys
Silicon	Ubiquitous; soils, sand, glass, concrete, building materials,   asbestos, printers, dry chemical fire extinguisher
Sodium	Fingerprints, humidification, soils, concrete
Sulfur	Outdoor air, combustion particles, cardboard, gypsum wallboard
Titanium	Paints, building materials
Zinc	Corrosion protection (galvanized and chromated zinc steels), paints,   building materials

 

 

Spectra of these samples revealed that certain elements were typically present in dusts in the building.  Magnesium, calcium, silicon, sulfur, and aluminum were typically observed, as were sodium, chlorine, potassium, and iron.  The elemental constituents detected in the samples were expected based upon typical urban pollutants and elements in common building materials, e.g., concrete/cement, wallboard, plaster, etc.   Significant peaks of calcium and oxygen in some samples is indicative of calcium oxide, attributable to pulverized concrete/cement present in the particulate that entered the facility following the collapse of the World Trade Center buildings, or was generated by damages to cement structures in the office.  A number of ESEM spectra, representing the various elemental particulate profiles observed in all samples, are presented as Spectra 1 through 26.  The elemental key for the observed elements is as follows: carbon (C), oxygen (O), sodium (Na), magnesium (Mg), aluminum (Al), silicon (Si), sulfur (S), chlorine (Cl), potassium (K), calcium (Ca), and iron (Fe).

 

Several systems on the 1^st, 2^nd and 4^th floors of the office sustained water damages that resulted in corrosion.  (Such systems included D4 Channel Banks, SLC Series 5, Telecom Solutions Clocks, Distributing Frames, and the Nortel DMS Urban Switch, SMAS, Fujitsu FLM and NEC muxes, fuse shelves, Lucent DACS, DSC CS1, etc., discussed in Sections 3.0 and 4.0.)  The particulars of the occurrence of corrosion are discussed in Sections 3.0, 4.0, and 7.0, which address the 2^nd, 4^th, 1^st floors, respectively.  Chemical, galvanic, and electrolytic corrosion were all observed within water-damaged equipment.  Where it occurs, blue-green electrolytic corrosion renders metallic conductors irreparable as due to the displacement of the base copper conductor metal below protective platings.  Corrosion at such sites will continue over time as the base copper is continually exposed to atmospheric moisture and oxygen.  Spectra 27 through 29 illustrate examples of corrosion residues, as indicated by the copper (Cu), nickel (Ni), and zinc (Zn) in the spectra.

 

 

Spectrum 1.  2^nd Floor – Cross-Connect Frame 10 (140 ft. from Washington Street wall)

 

 

Spectrum 2.  2^nd Floor – RR 2227 – Nortel FD-135

 

 

Spectrum 3.  2^nd Floor – RR2306.4 – NEC FD-31201A ~ 4ft. High

 

Spectrum 4.  2^nd Floor – RR2215.06 – top surface of MUX

 

 

Spectrum 5.  2^nd Floor – RR 236.14 – Lightguide FDF

 

 

Spectrum 6.  3^rd Floor – PBD 301 B.01 – AC Distribution Panel

 

 

 

Spectrum  7.  3^rd Floor – Power Room – Lucent Rectifier

 

 

Spectrum 8.  4^th Floor – RR4015.00 – DACS IV

 

Spectrum 9.  4^th Floor –RR 4416.03 – Tellabs Titan 5500 – Shelf 2

 

 

Spectrum 10.  4^th Floor – RR 4501.01 – Cross-Connect F75

 

 

Spectrum 11.  4^th Floor – RR 4519.08 – Fujitsu FLM-600

 

 

Spectrum 12.  4^th Floor – RR 4607.00 – Telecom Solutions DCD-523

 

 

Spectrum 13.  4^th Floor – RR 4601.15 – Fujitsu FLM-600

 

 

Spectrum 14.  6^th Floor – Power Room – Horizontal Battery Stand Surface

 

 

Spectrum 15.  6^th Floor – Power Room – PDB 601 6 – Lucent Rectifier G-9

 

 

Spectrum 16.  7^th Floor – DMS-100 ENET ENCI 00 Frame AA 17

 

 

Spectrum 17. 7^th Floor – DMS-100 E01 LGE 006 Horizontal Surface

 

 

 

Spectrum 18.  7^th Floor – 5ESS DS1 SM002 LTP2 Bulk Dust

 

 

Spectrum 19.  7^th Floor – 5ESS DS0 Cable Rack Near SM101

 

 

Spectrum 20.  8^th Floor – Distributing Frame – Vertical 78 ~ 4 Ft. High

 

 

Spectrum 21.  8^th Floor – Distributing Frame – Vertical 620 ~ 4 Ft. High

 

 

Spectrum 22.  9^th Floor Battery 5 – String Closest To Broken Windows

 

 

Spectrum 23.  9^th Floor – Cosmic Frame Shelf 19-2 Rear of Frame

 

 

Spectrum 24.  9^th Floor – DMS100 DS3

 

 

Spectrum 25.  9^th Floor – DMS-100 DTET 04

 

 

Spectrum 26.  9^th fl. – RR 9050A.10 – Winstar Collocation Space Demarcation Point

 

 

Spectrum 27.  4^th Floor – RR4004A.02E – Electrolytic Corrosion Fujitsu FLM-150 MUX Internal

 

 

Spectrum 28:  4^th Floor – R4200.14 – Electrolytic Corrosion SLC 5 BK 3

 

Spectrum 29.  8^th Floor – Electrolytic Corrosion – Distributing Frame – Vertical 172 ~ 4 Ft. High

 

 

 

Ion Chromatography

Telcordia collected 195 samples of ionic residue from various network equipment surfaces. These samples were collected by placing filter papers of known area wetted with deionized water onto metallic equipment surfaces brushed clean of bulk particulates.  As the papers dry, the ionic residues are absorbed onto the papers.  Back in our laboratory, the samples are returned to deionized water into which the ionic residues re-dissolve.  The resulting solution is then analyzed  for water-soluble ionic contaminants via ion chromatography (IC).  This analysis quantifies ionic contaminants that may have resulted from a particular event or were deposited over the lifetime of the equipment.  Chemical analysis of these residues determines the presence or absence of corrosive or hygroscopic compounds that can have adverse affects on framework and equipment components, leading to future reliability problems.  Samples were collected after removal of bulk particulates in order to best quantify to the actual amount of gas phase products of combustion (if any) and absorbed contaminants deposited on network equipment surfaces.

 

The results of the IC analyses are presented in Tables 8-3 through 8-7.  Results for each sample are expressed in units of micrograms per square inch (mg/in²).  A soiling rate of less than 2 micrograms per square inch per year (mg/in²/yr) for individual anions is common and the concentration for individual anions should remain below 50 mg/in² to ensure reliability.  Samples in red exceed 50 mg/in² concentrations for sulfate. “ND” denotes concentration below the level of detection of ion chromatography tests of 0.05 mg/in².

 

                 

TABLE 8-3    2ND FLOOR ION CHROMATOGRAPHY    RESULTS
NO.	LOCATION	CHLORIDE	NITRATE	SULFATE
T1	RR2013.03 D4 BANK 4	1.23	1.50	8.69
T2	RR2013.03 D4 BANK 2	1.60	1.19	20.4
T3	RR2013.06 D4 BANK 3	1.21	0.84	22.8
T4	RR2013.06 D4 BANK 1	1.17	ND	46.7
T5	RR2013.08 D4   BANK 3	22.5	1.49	73.1
T6	RR2013.08 D4   BANK 1	17.8	1.93	76.2
T7	RR2013.17 D4 BANK 5	1.05	ND	20.7
T8	RR2013.17 D4 BANK 1	1.03	ND	10.3
T9	RR2013.22 D4 BANK 4	1.44	ND	14.2
T10	RR2013.22 D4, BANK 2	1.42	1.00	28.4
T11	RR2305.04 SLC 5 SYS. #34	1.41	2.83	9.44
T12	RR2305.02 SLC 5 SYS. #11	1.34	9.51	14.3
T13	RR2305.01 DDM1000 SYS. 101	0.97	2.42	8.61
T14	RR2401.03 D4 BANK 3	1.01	1.16	7.86
T15	RR2400.02 D4 BANK 1	1.23	2.16	8.50
T16	RR2301.08 MUX 6	1.20	6.80	32.9
T17	RR2011.11 D4 BANK 3	1.01	ND	18.8
T18	RR2011.08 D4 BANK 4	1.08	0.80	13.0
T19	RR2011.05 D4 BANK 1	1.09	ND	9.09
T20	RR2011.03 D4 BANK 5	1.04	ND	37.2
T21	RR2010.01 D4 BANK 2	1.08	1.00	14.7
T22	RR2010.06 D4 BANK 3	1.08	0.92	15.5
T23	RR2006.01 D4	1.00	ND	8.12
T24	RR2010.10 D4 BANK 4	0.92	ND	13.4
T25	RR2006.06 D4 BANK 1	1.10	1.18	23.6
T26	RR2006.03 D4 BANK 4	1.14	1.95	21.0
T27	RR2006.09 D4 BANK 5	1.53	1.78	14.9
T28	RR2007.05 D4 BANK 2	1.26	1.41	14.5
T29	RR2004.02 MFT SYSTEM #51	1.41	1.05	16.9
R1	RR2101.12 D4 BANK 1	1.63	1.33	72.2
R2	RR2111.1 D4 BANK 1	2.48	0.96	15.3
R3	RR2313.09 D4 BANK 1   (CLEANED)	1.40	2.76	5.55
R4	RR2313.05 D4 BANK 1   (CLEANED)	28.0	4.86	47.6
R5	RR2213.12 D4 BANK 2   (CLEANED)	3.45	4.32	6.00
R6	RR2102.12 D4   BANK 2	2.32	ND	113
R7	RR2110.10 D4 BANK 2	1.51	2.44	30.4
R8	RR2111.10 D4 BANK 2	1.46	ND	9.07
R9	RR111.09 D4 BANK 3 (NEW)	1.31	2.31	12.6
R10	RR2213.16 D4 BANK 3   (CLEANED)	5.04	6.76	10.9
R11	RR2110.1 D4 BANK 3	1.27	2.12	14.4
R12	RR2313.05 D4 BANK 4   (CLEANED)	10.5	2.53	38.1
R13	RR2100.9 D4 BANK 5	2.33	1.91	40.1
R14	RR2213.12 D4 BANK 4	1.15	2.54	24.0
R15	RR2102.3 D4 (ZNCR)	1.82	ND	11.7
R16	RR2213.01 D4 MID	1.00	3.25	14.5
R17	RR2213.08 D4   BANK 5	23.6	4.08	61.4
R18	RR2101.4 MTCE 76248	1.85	1.50	55.6
R19	RR111.19  D4 (NEW)	1.27	2.45	8.27
R20	RR2213.01 D4   BANK 1	16.4	4.32	140
R21	RR2213.12 D4   BANK 1	16.8	2.43	53.9
R22	RR2213.04 D4   BANK 1	1.40	2.14	92.1
R23	RR2213.08 D4   BANK 2	1.41	2.73	64.2
R24	RR2111.7 D4 BANK 4	1.43	0.90	14.3
R25	RR2213.04 D4 BANK 5	1.58	0.30	3.01
R2-1	RR2228.8 WESTERN ELECTRIC   MX3 LTMA DS1	1.64	7.62	9.56
R2-2	RR2240.12 WESTERN ELECTRIC   CHANNEL UNITS	1.07	4.15	7.19
R2-3	RR235.2 WESTERN ELECTRIC   MX3 DS1, TOP	3.37	14.18	16
R2-4	RR235.2 WESTERN ELECTRIC   MX3 DS1, BOTTOM	5.85	7.27	15.1
R2-5	RR2228.9 TELO GTS MUX	5.15	4.31	16.5
R3-1	RR 2114.5 SMAS RELAY SHELF	1.80	2.38	27.3
R3-2	RR 2100.4 SMAS RELAY SHELF	1.39	1.87	16.6
R3-3	RR 2114.2 SMAS RELAY SHELF	2.59	1.19	16.8
R3-4	RR 2100.1 SMAS RELAY SHELF	1.81	2.51	14.9
Note 14% of   samples exceeded 50 mg/in2.

 

                 

TABLE 8-4    4TH FLOOR ION CHROMATOGRAPHY    RESULTS
SAMPLE	LOCATION	CHLORIDE	NITRATE	SULFATE
T2-1	RR4010.11 NEC MUX #4	1.13	1.26	3.08
T2-2	RR4010.09 NEC   MUX #2	1.69	1.47	71.1
T2-3	RR4301.12 ALCATEL 1633SX   SHELF 1	1.10	1.45	2.26
T2-4	RR4301.08 ALCATEL 1633SX   SHELF 1	1.77	1.33	4.46
T2-5	RR4301.04 ALCATEL 1633SX   SHELF 1	1.59	2.33	12.4
T2-6	RR4301.01 ALCATEL 1633SX   SHELF 3	1.21	ND	0.94
T2-7	RR4307.00 TELLABS K45, DS3,   SHELF 4	21.1	3.22	24.1
T2-8	RR4307.03 TELLABS K45, DS3,   SHELF 1	2.40	3.96	24.1
T2-9	RR4307.05 TELLABS K46, DS3,   SHELF 3	2.03	3.66	12.4
T2-10	RR4307.07 TELLABS K46, DS3,   SHELF 2	2.91	4.88	20.0
T2-11	RR4015.13 UP SWITCH BAY   DACS 1	6.51	4.40	48.2
R2-6	RR4416.04 TELLABS K46 SHELF   4	3.00	3.27	9.85
R2-7	RR4413.10 TELLABS K45 SHELF 3	1.58	2.48	8.02
R2-8	RR4416.04 TELLABS K46 SHELF   1	2.36	3.42	5.46
R2-9	RR4302.08 ALCATEL 1631   SHELF 1 TOP	1.67	4.28	3.91
R2-10	RR4303.01 ALCATEL 1631   SHELF 1	1.41	3.40	6.30
R2-11	RR4302.03  ALCATEL 1631 SX SHELF 1	2.17	13.9	20.2
R2-12	RR4413.06 TELLABS K46 SHELF   2	2.63	3.84	8.52
R2-13	RR4309.10 TELLABS SHELF 2	3.38	3.86	5.58
R2-14	RR4416.02 TELLABS K46 SHELF   3	1.89	3.74	36.0
R2-15	RR4416.00 TELLABS K46 SHELF   3	1.89	3.32	12.6
R2-16	RR4413.05 TELLABS K46 SHELF   4 TOP	1.95	4.52	2.52
R2-17	RR4302.10 ALCATEL 1631SX   DS1 SHELF 4	3.37	2.88	4.50
R2-18	RR4309.01 TELLABS K45 SHELF   7	2.21	4.00	11.0
R2-19	RR4003A.00 NORTEL S/DMS   SHELF A	2.64	1.99	15.2
R2-20	RR4003A.03 NORTEL S/DMS   SHELF C	8.08	1.85	40.5
R2-21	RR4014.08 DACS   IV SHELF C	3.33	5.69	92.8
R2-22	RR4013.03 DSC UNIT 4	2.69	1.89	24.0
R2-23	RR4013.08 DSC   UNIT 24	2.74	2.07	58.7
R2-24	RR4014.02   AT&T DACS IV K41	6.01	11.1	62.8
R2-25	RR4019.16 DSC   DSN ACCESS	2.39	2.88	74.9
R2-26	RR4019.11 DS3 RACK	1.26	2.29	4.96
R2-27	RR4200.11 SLC5	1.42	1.82	5.45
R2-28	RR4200.08 SLC5	2.88	2.42	30.4
R2-29	RR4018 DSC   DEXCS DSN A (END OF LINEUP)	1.88	2.97	80.2
R2-30	RR4019.00 DSC   ADMIN	3.60	4.09	85.6
R2-31	RR4309.05 TELLABS K46	1.65	3.75	43.9
R3-5	RR 4402.09 D4 BANK 1	1.08	2.98	7.20
R3-6	RR 4401.13 D4 BANK 2	4.75	9.94	10.2
R3-7	RR 4402.05 D4 BANK 3	1.80	3.24	10.1
R3-8	RR 4402.00 D4 BANK 4	1.71	2.99	10.9
R3-9	RR 4401.13 D4 BANK 6	4.90	5.35	11.7
R3-10	RR 4010.08 NEC FD-1840A   MUX4 OFF DUST	1.38	1.52	13.6
R3-11	RR 4010.08 NEC FD-1840A   MUX4 ON DUST	5.31	4.87	2715*
R3-12	RR 4007.05 E FLM-150 ADM	2.44	1.42	18.4
R3-13	RR 4200.03 SLC 5	1.16	1.75	7.14
R3-14	RR 4201.05 SLC 5	1.32	3.52	7.13
T4-1	RR4903.08 CISCO 7500 ROUTER	1.76	3.50	ND
T4-2	RR4904.00 MAX MODEM	2.78	6.16	ND
T4-3	RR4818.04 TELLABS TITAN 5500   SHELF 2	2.01	2.85	ND
T4-4	RR4816.07 TELLABS TITAN   5500 SHELF 4	1.60	4.04	ND
T4-5	RR4606.01 FUJITSU FLM 600   SHELF D	0.98	ND	ND
T4-6	RR4608.01 LUCENT WAVESTAR   SHELF B	1.04	1.09	ND
R4-1	RR4601.00 NEC RC-28D MUX	1.28	1.48	4.04
R4-2	RR4803.03B FUJITSU FLM 2400	1.17	1.33	ND
R4-3	RR4513.14 DSLAM SYS. 4 CH5   SHELF C	1.24	1.17	ND
R4-4	RR4606.15 FUJITSU FLM 2400   SHELF A	1.07	ND	ND
R4-5	RR4804.11 FUJITSU FLM   600/2400	1.47	1.01	1.72
R4-6	RR4513.03 DSLAM SYS. 1 CH3   SHELF B	3.15	1.18	6.23
S-1	RR 4003.08 NORTEL URBAN   HIGH SHELF INTERIOR VERTICAL ZNCR	1.02	ND	9.26
S-2	RR 4003.05 NORTEL URBAN   HIGH SHELF INTERIOR VERTICAL ZNCR	1.33	ND	14.1
S-3	RR 4003.03 NORTEL URBAN LOW   SHELF VERTICAL ZNCR	3.51	ND	28.7
S-4	RR 4003.00 MID SHELF   INTERIOR VERTICAL ZNCR	1.49	1.30	9.13
S-5	RR 4000.06 MID SHELF HEAT   SINK	1.06	ND	11.2
S-6	RR 4000.03 MID SHELF HEAT   SINK	1.66	ND	10.7
S2-1	RR4103.01 DSC IMTN DACS BDM   A1-3	1.12	ND	9.95
S2-2	RR4104.03 DSC IMTX DACS BDM   B3-3	1.62	ND	2.83
S2-3	RR4103.06 DSC IMTX DACS BDM   A6-3	3.22	ND	7.31
S2-4	RR4104.08 DSC IMTX DACS K33   BDM B8-3	1.39	ND	6.97
S2-5	RR4106.01 DSC IMTX DACS HSO   12	1.13	ND	2.95
S2-6	RR4105.09 DSC IMTN DACS K33   HSO 64	1.36	ND	2.06
S2-8	RR4113.03 DSC DEXCS	1.31	ND	2.08
S2-7	RR 4115.02 TELLABS TITAN   5500 TOP SHELF STAINLESS STEEL	1.02	ND	1.27
S2-9	RR4012.04 DSC DEXCS	1.46	2.37	2.46
S2-10	RR4515.10 LITESPAN CBA1	0.99	ND	ND
S2-11	RR4523A.02 FLM 150 SHELF 4	1.14	ND	ND
S2-12	RR4523.10 FLM 150 SHELF 4	1.05	ND	2.99
Note 9% of samples   exceeded   50 mg/in2.

* Note that the high result for Sample R3-11 was sampled directly on dust and is indicative of the composition of the dust rather than gas-phase deposition of ionic contaminants.

 

                 

TABLE 8-5    6TH FLOOR ION CHROMATOGRAPHY    RESULTS
SAMPLE	LOCATION	CHLORIDE	NITRATE	SULFATE
R5-1	6TH   FLOOR POWER ROOM LUCENT RETIFIER G-1	3.91	18.1	58.4
R5-2	6TH FLOOR POWER   ROOM LUCENT RETIFIER G-9	3.33	22.6	17.9
R5-3	6TH FLOOR POWER   ROOM PBD 601.00 LORAIN/RELIANCE FUSE PANEL	3.05	17.0	18.5
Note 33% of samples   exceeded   50 mg/in2.

 

                 

TABLE 8-6    7TH FLOOR ION CHROMATOGRAPHY    RESULTS
SAMPLE	LOCATION	CHLORIDE	NITRATE	SULFATE
S-7	5ESS DS1 SM001 LTP 0 SHELF   62 1986	4.67	4.81	10.7
S-8	5ESS DS1 SM34 SMC1 SHELF 62   1986	2.49	1.54	6.60
S-9	5ESS DS1 SM56 LTP0 SHELF 45   1988	5.43	0.80	3.90
S-10	5ESS DS1 SM81 LTP0 SHELF 45   1993	2.13	3.59	10.8
S-11	5ESS DS1 SM 101 LTP0 SHELF   95 1996	1.46	3.20	8.30
T3-1	5ESS DS2 SM018 LTP2 SHELF   53 1987	2.08	1.50	7.36
T3-2	5ESS DS2 SM038 SMC1 FUSE   SHELF	2.45	1.50	1.13
T3-3	5ESS DS2 SM056 LTP0 LINE   GROUP A	1.57	1.74	11.0
T3-4	5ESS DS2 SM003 SHELF 36 LTP0	7.45	4.02	15.1
T3-5	5ESS DS2 M04 UNIT RA05	1.19	0.81	5.07
T3-6	5ESS DS2 SM032 SHELF 62   SMC1	1.90	1.12	2.23
T3-7	5ESS DS2 SM055 SMC1 SHELF   36	1.62	0.91	1.33
R3 -15	DMS LGE 002 E03 REAR   BRACKET	6.76	5.37	39.6
R3 -16	DMS SME 002 E16	11.5	5.55	10.5
R3 -17	DMS LGE 011 G07 FAN COOLED	1.22	2.47	1.78
R3 -18	DMS LCE 047 LCA 1 REAR   BRACKET	2.25	7.11	3.09
S3-1	5ESS DS2 SM057 LTP002 SHELF   36 (FANS) 1997	0.90	ND	ND
S3-2	5ESS DS2 SM040 LTP0 SHELF   53	1.89	ND	2.45
S3-3	5ESS DS2 SM014 SMC1 SHELF   53	1.04	ND	1.09
S3-4	5ESS DS2 DSC 03 SHELF 28	1.19	ND	1.39
S3-5	5ESS DS1 CM2 1 99	1.98	1.10	ND
S3-6	5ESS DS1 SM63 SMC1 SHELF   45  89	3.91	1.55	1.72
S3-7	5ESS DS1 SM076 LTP2 SHELF   53  91	1.44	ND	0.66
S3-8	5ESS DS1 SM105 SMC1 SH53 96	1.49	ND	0.82
R5-4	DMS DTEI 15 H22 LOW  9/2000	1.94	3.96	7.94
R5-5	DMS DTEI 15 H22 HIGH  9/2000	1.36	4.29	1.34
R5-6	DMS SME 004 J03 1995	4.67	15.5	4.48
R5-7	DMS SME 010 F22 5/2001	1.00	1.84	ND
R5-8	DMS LCE 004 D02	4.81	6.92	6.32
R5-9	DMS LGE 009 H11 1996	1.95	3.29	1.55
R5-10	DMS LCE 029 H04	2.11	2.93	2.94
R5-11	DMS ENCI 00 ENET1 AA17   INSIDE DOOR	1.13	2.34	ND
Note 0% of samples   exceeded   50 mg/in2.

 

                 

TABLE 8-7    9TH FLOOR ION CHROMATOGRAPHY    RESULTS
SAMPLE	LOCATION	CHLORIDE	NITRATE	SULFATE
R3-19	DMS DTES-04 J10	1.81	2.16	1.53
R3-20	DMS LGE 004 B07  1994	4.37	8.67	4.22
R3-21	DMS MT7X76BA ADJ DTE017 E10   1995	3.76	3.63	3.29
R3-22	DMS TME 014 F03 1994	3.17	5.95	5.08
R3-23	DMS LGE 11 J02 2FT IN FRONT   OF BROKEN WINDOW	3.57	6.6	8.26
R5-12	DMS LGE 02	8.26	4.28	4.89
R5-13	DMS LGEI 000 S02	4.25	2.10	4.88
R5-14	DMS SMEI 000 1995	4.91	4.67	3.14
R5-15	DMS DTE 12 01	3.67	2.42	3.42
R5-16	DMS DTE 025	6.03	3.36	47.6
R5-17	DMS DTE 28 T09 (LOW) 1997	2.63	2.59	7.97
R5-18	DMS DTE 28 T09 (HIGH) 1997	2.41	2.94	10.2
R5-19	DMS DTE 31	3.52	7.58	5.49
R5-20	POWER ROOM AT&T LINEAGE   G14 EXTERNAL	3.73	6.26	6.43
R5-21	PDB 903.01 LORAIN AC-DC	3.28	12.6	6.24
R5-22	POWER ROOM AT&T LINEAGE   G2 INTERNAL	2.03	3.29	1.16
S3-9	DMS DPCC0 INSIDE ZNCR DOOR	1.26	1.66	1.13
S4-1	COSMIC HVAC DUCT FARTHEST   FROM MAPS -GALVANIZED – HORIZONTAL	1.74	2.54	17.8
S4-2	COSMIC RM ZNCR CABLE RACK   HARDWARE VERTICAL	8.74	6.56	9.25
Note 0% of samples   exceeded   50 mg/in2.

 

The primary ionic constituents detected by the IC analyses were chloride, nitrate, and sulfate.  No fluoride, bromide, phosphate, or sulfite were detected or were detected in de minimis concentrations in isolated instances.  None of the results for chlorides or nitrates exceeded Telcordia criteria.  Sulfate data exceeds the 50 mg/in² criteria in localized areas of the 2^nd, 4^th, and 6^th floors.  However, no consistent spatial relationship exists between locations where excess sulfate was detected.  Telcordia believes that these exceedances represent sulfate in dust captured on the sampling media rather than gas phase contaminants deposited directly onto equipment surfaces.  The low percentages of exceedance indicate that the gas-phase deposition of ionic contaminants is not a substantial concern for the network equipment, and that no remedial action is required to address ionic contamination.

 

pH Measurements

pH measurements of residues were made throughout the 2^nd, 4^th, 7^th, and 9^th floors of the facility, and exhibited values of 12 to 14. A value of less than 7 is acidic, indicating excess H₃O⁺ ions. 7 is neutral, and a value greater than 7 is alkaline, indicating excess OH^– ions.   The pH values detected are consistent with the expected debris from the destruction of cement, wallboard, and other building materials.  These residues would become corrosive to metallics if they were to become damp or wet.

 

Copper Mirror Corrosion

Particulate samples entrained into the facility as a result of the event were individually dissolved in deionized water and placed on a copper mirror along with a control deionized water sample. The degree of the corrosivity is based on the reaction time, i.e., the time it takes for the solution to corrode through the thin copper film. All test samples showed impact to the copper mirror after ~1 hour, indicating corrosivity.

 

Insulation Resistance Test

One concern with particulate contamination of circuit pack electronic equipment is the potential for particulates to be hygroscopic, that is, to getter moisture from ambient air, thereby becoming wet and conductive.  Hygroscopic dusts, when wet, result in shorts and shunts when particulates bridge across normally isolated conductors on circuit packs, backplanes, and connectors.  In order to determine how hygroscopic the dust from the September 11 event is, Telcordia conducted an insulation resistance test on samples of particulates from the second floor of the office.  The test involves placing particulate on a test coupon with conductors of a known spacing, and then applying a –48 volts dc potential in a test chamber where the relative humidity is increased from 20 percent to 98 percent.  The results are logged and compared to a reference coupon.  The graph below illustrates the results of the test.

 

Review of the data indicates that the surface resistance of the dust collected from the office dropped approximately two orders of magnitude (approximately 4×10¹¹ ohms to 2×10⁹ ohms) at humidity levels above approximately 70 percent (pink and yellow).  The result for a blank test coupon was similar (aqua).  Superimposed on the graph (dark blue) is the measured result for a control coupon contaminated with particulate that becomes hygroscopic at approximately 70 percent relative humidity.  This particulate exhibits a five order of magnitude drop in resistance.  The data indicate that the particulate from the office is not hygroscopic. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 9.0

Vibration Concerns

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Summary of Conclusions

 

·         Restoration of structural elements of the 140 West Street office will likely generate vibrations in excess of network equipment design limits.

·         Telcordia strongly recommends that Verizon engage subject matter experts to review the proposed work, set vibration limits for the project, specify monitoring points and systems, provide or oversee real-time vibration monitoring, and provide guidance on work methods to mitigate vibration concerns.

 

Vibration Concerns Associated With Structural Repairs

 

Evaluation of the structural damages sustained at 140 West Street indicate that substantial structural repairs will be needed.  Foreseeable repair operations will include the erection of temporary exterior walls, demolition of damaged exterior walls and floor slabs, replacement of building columns and laterals, and subsequent construction of new walls and slabs.  Building restoration projects involving comparable work will usually be conducted using vibration intensive tools, including jackhammers, impact drills and hoe rams, etc. Some vibration-intensive activities will need to occur in extremely close proximity to existing operating network equipment, and unless addressed, will generate transient vibrations in excess of the equipment design levels.  These vibrations pose a serious threat to the integrity of operations and long-term reliability of nearby network equipment.  Risks from vibrations include data errors or head crashes on hard disk drives, circuit pack unseating problems and resultant arcing, inadvertent opening or closing of sensitive electro-mechanical relays, battery shifting “walking”, or in extreme cases, physical damages to hardware including cable racks and cable insulation. Raised floors, such as those used in switch installations on the seventh and ninth floors, can amplify lower frequency vibrations.

 

A number of methods are available to mitigate vibration risks, including engineering controls, alternate work methods, and real-time monitoring of the vibrations produced during all phases of restoration activities. Engineering controls would include the isolation of structural components of the building or other structural modifications to dampen or isolate network electronics from the vibration source(s). Specification of alternative work methods can reduce vibration, such as the use of lighter duty equipment, alternative demolition and construction techniques, and the specification of minimum operating distances between restoration operations and network equipment.  Subject matter experts can make such specifications after review of demolition and construction plans, test and review of operating equipment to remain, and real time monitoring of vibration assessment equipment. Vibration monitoring by experienced subject matter experts would allow restoration work to be stopped and altered before damaging levels of vibration were sustained, and can be supplemented by specification of alternative methods.

 

Several resources are available that provide guidance regarding vibration generated by demolition and construction operations.  These include Telcordia SR-3452 – Vibration Effects of Building Demolition, Telcordia SR-3751 – Guidelines For Vibration Assessment And Control-Physical Protection And Network Hardware, and Telcordia NR-NTA-02016 – 8 –02 –001 – Vibration Tolerance of Network Equipment.  Table 9-1 below provides general criteria regarding maximum vibration levels for network equipment, measured at the floor.  Figure 9.1 below illustrates the equipment limits for transient vibrations with a duration of one second or less.  Figure 9.2 below illustrates the vibration limits for building structural elements.  Additionally, subject matter experts in Telcordia are available for project-specific guidance.

 

TABLE   9-1 GENERAL   VIBRATION LIMITS FOR NETWORK EQUIPMENT
System Type	Source	Characteristics	Parameter	Criteria
General Equipment	Continuous Vibrations	Broad Frequency Content	Displacement	£0.001 in. peak to peak
			Acceleration	£0.15g
		Single Dominant Frequency	Acceleration	£0.03g
	Transient Vibrations	—	—	See Figure 9.1 below.
Vibration Resistant   Equipment	Continuous Vibrations	Broad Frequency Content	Displacement	£0.01 in. peak to peak
			Acceleration	£0.5g
		Single Dominant Frequency	Acceleration	£0.1g
	Transient Vibrations	—	—	See Figure 9.1 below.

 

 

Figure 9-1 – Transient Vibration Levels for Building Floors Supporting Telecommunication Equipment

 

 

Figure 9-2.  Transient Vibration Levels for Building Structural Elements

 

Telcordia recommends that vibration monitoring be conducted to ensure that the levels above are not exceeded in equipment areas during facility restoration.  Vibration limits should be set for the restoration work at levels conservatively below the criteria discussed above.  If vibration levels are found to exceed conservative project limits, work methods should be adjusted to reduce vibration levels, or the work should be halted until such adjustments can be resolved.  Given the criticality of this office, the scope and complexity of its network systems, and the extensive structural damages requiring repair, Telcordia strongly recommends that Verizon and its contractors review the proposed work, set vibration limits for the project, specify monitoring points and systems, monitor vibrations, and provide guidance on work methods.   Telcordia subject matter experts are available to assist Verizon and its contractors in all phases of vibration management mitigation matters.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 10.0

Thermal Concerns

 

 

 

 

 

 

 

 

 

 

 

Conclusions

• The two Lucent 5ESS switches (DS-1 & DS-2) located on floor 7 and the 9^th floor Nortel DMS-100 experienced loss of environmental control coupled with heavy frame filter loading and thermal blanketing of circuit pack components.
• Temperature measurements with simulated frame filter loading and calculation revealed higher heat dissipating frames, e.g., switch processor (PCCA & DPCC) frames, would operate approximately 44°F above ambient and exceed intended design criteria if ambient conditions top 110°F.
• Thermal blanketing of board mounted electronic components would further reduce heat transfer resulting in more severe thermal and electrical stresses raising performance and warranty concerns.

 

Telecommunications network equipment deployed in Verizon facilities are designed to operate within defined environmental conditions. Lucent 5ESS digital switch, Nortel DMS-100 digital switch, DAC IV systems, transport equipment, etc., meet Telcordia NEBS requirements for temperature as specified in GR-63-CORE, Network Equipment Building System (NEBS): Physical Requirements Section 4.1.2. The high temperature requirement is for continuous reliable equipment operation up to a space temperature of 104°F, with short-term exposure to 122°F for 96 hours. These space temperatures are measured in equipment frame line-up aisles 15 inches (380 mm) in front of the centerline of the frame at a height of 59.1 inches (1.5 meters) above the floor.

 

It is important to note that these high temperature performance criteria are based on new equipment without excessive electronic component soiling or filter loading, fan failure, environmental aging, or other impediments that are not conducive to proper operation.  With higher heat load equipment such as digital switch processor frames, fan operation and adequate airflow are necessary to maintain acceptable internal frame temperatures.  Table 10-1 below provides the manufacturer specified heat output in watts and watts per square foot for several types of Lucent 5ESS, Nortel DMS-100, and Lucent DACS IV frames found in 140 West Street.

 

Table 10 -1.  Heat Dissipation Specifications
Equipment   Type	Total   Heat Dissipation	Heat   Dissipation Watts/FT2
5ESS
SMC	755   watts	72.5   watts/ sq. ft
LTP	600   watts	57.6   watts/ sq, ft
CM	1771   watts	170   watts/ sq, ft
PCCA	5243   watts	206.3   watts/ sq. ft
DMS-100
LGE	980   watts	178.9   watts/ sq. ft
DTE	1120   watts	124.5   watts/ sq. ft
DPCC	3500   watts	206.9   watts/ sq. ft
LCEI	1610   watts	178.9   watts/ sq. ft
TME	480   watts	54   watts/ sq. ft
LTE	1120   watts	124.5   watts/ sq. ft
LCE	1050   watts	116.7   watts/ sq. ft
DACS IV
DS1   frame	1750   watts	78.8   watts/ sq. ft
DS   1 Interface	560   watts	75.6   watts/ sq. ft
STS1/DS3   Interface	1144   watts	154.5   watts/ sq. ft

The DMS-100 switch located on the 9^th floor and the two 5ESS switches located on the 7^th floor sustained severe particulate contamination that blocked frame filters impeding normal airflow through the frames. In addition, many circuit board mounted components were blanketed by particulate, further impeding heat dissipation. To determine the effect of reduced airflow in these frames, real time temperature measurements were conducted within each shelf of two DMS-100 switch frames that are operating in our Red Bank, NJ laboratory.  The frame filters were blocked approximately 90% to simulate the severe frame filter loading experienced in the field following the September 11 event. The frames tested were DTE and LGE frames with heat dissipation of approximately 124 watts/ft² and 178watts/ft², respectively.  As expected we see a higher temperature increase in the hotter frame. Figure 10-1 below presents the increase in temperature above ambient room temperature at 5-minute intervals.

 

Figure 10-1

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Under normal operating conditions (time 0) the measured internal frame temperatures were 3°F and 11°F above ambient.  With frame filter loading the internal temperature continues to increase until it reaches thermal equilibrium after approximately 50 minutes, i.e., the temperature rise is capped at 15°F and 33°F by the ability to dissipate heat via natural convection and thermal conduction through framework and cabling.

 

The above results were used to project internal temperatures for higher heat load frames, e.g., processor frames. Table 10-2 presents an estimation of the internal temperatures of several common fan-cooled DMS-100 and 5ESS switch frames with 90% frame filter blockage at different ambient room temperatures.  Note that the projections have some error due to non-linearity associated with specific frame airflow design characteristics and fan performance curves, but provides a reasonable approximation. Detailed modeling for specific individual frames can be performed if desired. This analysis references the measured temperature increase data presented in Figure 6-1 along with approximate heat dissipation numbers from Table 1.  The equipment includes DMS-100 DTE frames (approximately 125 watts/ft²), DMS-100 LGE, LCEI, and 5ESS CM frames (170-180 watts/ft²), and 5ESS PCCA and DMS-100 DPCC frames (207 watts/ft²).  As expected, the higher power frames becomes the hottest, exceeding 150°F when ambient conditions reach between 110 and 120°F.  Although internal temperatures guidelines are not specified for operating equipment, they should not exceed the maximum transportation and storage temperature for the components of 158°F as shown in Telcordia GR-63-CORE Table 4-2 High Temperature Thermal Shock. If switch room temperatures exceeded 110°F, we project that high heat load frames with clogged frame filters exceeded the design limit for temperature.

 

Table 10 – 2. Estimated Internal Frame Temperature @ 1 hour (°F) operation with 90% filter   blockage.
Ambient	DMS   DTE	DMS   LGE/LCEI & 5ESS CM	DMS   DPCC & 5ESS PCCA
°F
70	85	103	114
80	95	113	124
90	105	123	134
100	115	133	144
110	125	143	154
120	135	153	164

 

 

The effect of the elevated temperatures will be more severe due to thermal blanketing of electronic components by heavy dust. Dust that accumulates on surfaces and between board mounted electronic components interferes with normal radiated heat transfer from these components and will generate localized hot spots on integrated circuit chips, power modules/converters, resistors, etc. The cumulative effect of elevated ambient temperatures, significant blockage of fan filter media, and thermal blanketing of electronics is likely to cause some components to fail, particularly any marginal components with previous physical degradation, environmental exposures, and components nearing end-of-life.

 

The 7^th floor DMS-100 space temperature was elevated, however, the lack of contamination on fan filters and circuit pack components permitted normal airflow through the switch frames.  The elevated space temperature, although undesirable, did not exceed intended short-term design criteria per Telcordia GR-63-CORE.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 11.0

Cleaning Procedures for Asbestos Containing Particulate

 

 

 

 

 

 

 

 

 

 

 

 

Introduction

 

This procedure is intended as a general reference for undertaking the cleaning of D4 channel bank equipment that includes associated cable racks, hardware and adjacent facility. This equipment and facility have been contaminated with particulate that includes asbestos fiber, wallboard and concrete dust, as well as common soils. Note that the D4 equipment contains sub-assemblies of circuit packs, connectors and cabling that also require a detailed level of cleaning.  To clean the D4 channel banks, person(s) knowledgeable in the disassembly, functionality, trouble-shooting and powering requirements must be available during all phases of the D4 cleaning operation. This should include procedures for de-powering and removal and cleaning of the hard- wired PDU circuit packs prior to commencement of operation and additional power wire rear terminating screws and PDU’s should be on hand. Details must also be provided to remove and restore customer service and Telcordia assumes that all cleaning will commence during maintenance window hours.  Telcordia Technologies Disaster Prevention and Recovery Group welcomes Verizon and their contractor(s) questions regarding specific details of this document. Please read the entire document before proceeding with cleaning operations.

 

Equipment Needs

 

• Personal Protective Equipment (PPE) and personal training as specified for the 140 West Street Central Office by Verizon and all applicable Local, State and Federal laws.
• Two HEPA-filtered negative air machines equipped with 0.3 micron filtration and a minimum of 25 feet of clean, insulated, large diameter intake duct, and insulated duct for routing exhaust stream.
• Electrostatic Field (ESF) meter. A suggested scale is 0 to 20 kV/inch.
• Static safe portable workstation. Equipped with HEPA filtered chamber, ESD dissipative/grounded mats on table and floor.
• HEPA filtered vacuum cleaner. This must have an insulating hose section where it is connected to the ESD vacuum cleaner brush such that the ESD brush ground and the AC motor ground are not connected to the same electrical point. (See separate, attached vacuum cleaner specification.)
• ESD vacuum cleaner brush. The brush must be grounded to the equipment ground plane and be isolated from the AC ground plane.
• ESD wrist straps and controls for personnel.
• Extra circuit packs of the same codes as the packs being vacuumed.
• Circuit pack carrying case designed to carry a number of packs over short distances. These may be available from the equipment manufacturer.
• If required, portable room humidifier(s). Depending on the size of the office, outside temperature and relative humidity (RH), and rate of air exchange; multiple humidifiers may be required.
• Portable RH meter. This should be located at the work site where the pack vacuum cleaning is being done. This may not be the centralized location of the central office RH sensor.
• Electronic grade tack cloths.
• Compressed air or nitrogen.
• Extension cords with ground lead

 

 

General Considerations

 

Much of the particulate at the 140 West Street Central Office has been found to contain asbestos.   Personnel protective equipment (PPE) and safety training are required to perform this work and Telcordia strongly recommends that PPE and training be implemented per all Verizon and applicable Local, State and Federal requirements.

 

In conducting particulate cleaning, there is a risk of component damage from ESD (electrostatic discharge) when individual circuit packs are cleaned, but with proper precautions, cleaning can be accomplished successfully. The circuit packs should be carefully removed from the frames with minimal vibration and taken to an anti‑static work area (a bench with a static dissipating surface or pad). Exercise precautions to ensure that particles do not contaminate the electrical contacts on the backplane connector. It is intended that all circuit packs from a particular shelf be removed, and to expedite the cleaning process, the internal framework surfaces, the card cage and the circuit pack connector side of the backplane be cleaned simultaneously. Procedures for cleaning circuit packs, internal and external equipment surfaces are provided in the following sections.

 

Facility Cleaning

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2.
2. Cleaning should only be undertaken by trained workers familiar with operating in an asbestos contaminated environment that houses sensitive network electronics.
3. Cleaning should proceed from higher surfaces to lower surfaces.
4. Carefully clean all horizontal and vertical facility surfaces (ceiling, cable supports, cable, cable trays or racks, lighting, unistrut, wall-mounted equipment, floors, etc.) using a HEPA vacuum that meets industry standard asbestos remediation guidelines and the criteria:
   1. Cleaning must be done in a manner to minimize the distribution of dust or debris onto or into network equipment.
   2. Workers should always point vacuum exhaust away from uncleaned surfaces.
   3. c.       Vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit the re-contamination of equipment or facility surfaces.
   4. Following cleaning of a particular surface inspect the surface for areas of remnant dust.  Re-vacuum as necessary before proceeding to the next work area.

 

Front Of D4 Equipment

 

Note that the cleaning of electronic equipment normally requires that the space relative humidity remain between 40 and 55%. However, due to concerns with corrosion we recommend that the humidity be held between 25 and 45% during the 2^nd floor cleaning procedure.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
2. Locate one negative air machine equipped with 0.3 micron filtration at each end of the line up to be cleaned.  Ensure that the exhaust stream is directed away from uncleaned areas.
3. Obtain power for the negative air machine from house power, not outlets on the equipment frames.
4. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
5. Suspend one intake duct adjacent to each side of the frame shelf to be cleaned.
6. Start the negative air machine.
7. Obtain power for the HEPA vacuum from house power, not outlets on the equipment frames.
8. After grounding the operator and ESD brush of the vacuum to the equipment ground plane, vacuum the top and the front of the top bank of equipment working downward. Care must be taken to avoid changing any toggle equipment or push button settings on circuit packs.
9. Place blotter paper below the top channel bank to catch falling debris. 
10. Record the location and model codes for each pack in each shelf of the bank.
11. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)
12. Using ESD protection, unlatch and begin to pull a circuit pack for its slot.  After pulling the pack, vacuum bulk dust from component and solder-side surfaces and place in a static safe circuit pack carrier or bag.
13. WARNING ‑ When a pack is removed from the frame, contaminants on or around it will become airborne. It is important to maintain the vacuum in a position consistent with capturing the loose particulate. In addition, placing a piece of blotter paper under the pack to be removed will help minimize fallout.
14. Continue removing and vacuuming until one entire shelf of circuit packs is complete.
15. Periodically, check the level of the ES field at the brush to ensure that it is less than 2.5 kV/in.
16. Using the static-safe carrier, move the circuit packs to a grounded workstation for thorough cleaning. To expedite the cleaning procedure, this operation should be done by a dedicated individual(s) as outlined on page 1-5, under the circuit pack cleaning instructions of this document.
17. From the front, vacuum all accessible internal framework and card cage surfaces, both vertical and horizontal.
18. Vacuum the front of the backplane.  Carefully vacuum the connectors on the front of the backplane. Care must be taken to avoid any deposition of the vacuum brush fibers across pins or connectors should a fiber come loose from the brush. When a particular shelf is completed, verify that no remnant dust have been left behind.
19. If vacuuming does not remove all of the dust, the framework and circuit pack cage may be further cleaned by using a tack cloth. Do not use tack cloths on the internal connector surfaces of backplane connectors.
20. Replace circuit packs.
21. Reposition the air ducts of the negative air machine to the next lower shelf.
22. Moving downwards, repeat steps 11 to 19 on the individual shelves until all of the circuit packs and shelves in the frame are cleaned are cleaned.
23. Vacuum the floor under and around the frame and any remaining framework.
24. When cleaning is complete, return system to service, observe indicator lights and if necessary perform appropriate diagnostics.

 

 

 

Rear of Equipment

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
2. Locate one negative air machine at each end of the line up to be cleaned.  Ensure that the exhaust stream is directed away from uncleaned areas.
3. Obtain power for the negative air machine from house power, not outlets on the equipment frames.
4. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
5. Suspend one intake duct adjacent to each side of the bank to be cleaned.
6. Start the negative air machine.
7. Obtain power for the HEPA vacuum from house power, not outlets on the equipment frames.
8. After grounding the operator and ESD brush to the equipment ground plane, vacuum the rear surface of the plastic backplanes protective covers and remove.  Vacuum the internal side of the plastic cover and set aside.
9. Vacuum all immediately accessible vertical frame surfaces.
10. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)
11. Starting at the top of the bank, begin to vacuum all cables, wires, connectors, and adjacent surfaces.  If possible, a sheet of blotter paper should be held under the area being vacuumed to catch falling debris. Wires and cables should be spread as much as possible to gain access for vacuuming. For vertical cables, it will be necessary to cut the lacing cord to access all areas. These cables must be re-laced following testing/cleaning in a manner consistent with industry practices. Care must be taken to avoid any deposition of the brush fibers across pins or connectors should a fiber come loose from the brush.
12. Vacuum the rear of the backplane pin fields taking extreme care not to bend or deform the pins.
13. Periodically, check the level of the ES field at the brush to ensure that it is less than 2.5 kV/in.
14. After the cleaning of the back of the bank is complete, the backplane should be thoroughly inspected for remnant dust. Any dust should be re-vacuumed or wiped off with a tack cloth except on pin and connector surfaces. Brush fibers should be carefully removed by hand.
15. Replace the protective backplane covers.
16. Repeat Steps 5 to 14 on each shelf to clean the entire frame, top to bottom.
17. Vacuum the floor and any remaining framework.
18. Repeat procedure for each frame.
19. Vacuum all remaining floor space within the equipment room including all work and terminal areas.

 

 

 

 

 

Circuit Pack Procedure

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
2. Adjust the relative humidity of the circuit pack cleaning space (or the room where the packs will be cleaned if it is another room) to over 25% RH and less than 45%. This RH should be measured at the location where the packs are being cleaned.
3. Verify that at least one spare circuit pack of each type of pack to be cleaned is available on site.
4. 4.      Set up the filtered static safe workstation, vacuum cleaner head, and personnel grounding system.  These three points should be connected to a common ground. Under no circumstances should you use the same ground as the AC vacuum cleaner or negative air machine motor.
5. Using ESD protection, remove the packs to be cleaned from the shelf. Move the packs to a grounded, HEPA-filtered workstation. The person’s wrist strap performing the work and the ESD vacuum brush ground strap must be connected to the same ground point as the static safe ‑ grounded work station.
6. Run vacuum cleaner for at least three minutes. The vacuum cleaner AC power should be from the building power, and should not be contained from the equipment frame.
7. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)
8. Vacuum the circuit pack on both sides. 
9. Inspect the pack for cleanliness.  If dust of debris remains around hard-to access components, place circuit pack back in HEPA-filtered work chamber and blow off debris with compressed nitrogen or air (~15 psi), then re-vacuum the pack using the ESD vacuum brush.  Verify that no bristles from the vacuum have been caught on the pack components.
10. Vacuum the workstation cleaning surface.
11. Proceed to the next pack and start over with Step #7 of this procedure.
12. When a shelf of packs is completed, and the framework cleaning has been accomplished for this shelf, re-insert the packs in their original channel bank shelf paying close attention to the specific location.

 

 

HEPA VACUUM CLEANERS FOR CLEANING

ELECTRONIC EQUIPMENT

 

Shop vacuums, industrial vacuums, home and office vacuums, wet and dry vacuums, etc., that are designed for general use are not acceptable for telecommunications equipment applications. They are usually not efficient enough to handle submicron particle sizes, often generate ESD, often are not shielded for EMI, and may raise humidity levels.

 

To be used successfully on electronic equipment, the units should have the following characteristics:

1. The unit should have multilevel filtering with the final stage being a high efficiency particle arrestance (HEPA) filter, with a rating of at least 99.97% removal efficiency for particles down to 0.3 micrometers in diameter or larger.
2. Wet scrubber vacuum systems should not be used in central offices due to concerns over suddenly raising humidity levels, inefficiency of filtration, and potential spills.
3. Hoses and wands should be of sufficient length to allow cleaning of cable racks and under the base of electronic equipment; 25 feet is suggested. Hoses and any components, which may come into contact with electronic equipment, must be insulating.
4. Brush fiber lengths should be of the order of 5 cm. (2 inches) to allow cleaning action around and through cabling, wiring, channels, etc.
5. Power cords should be of sufficient length to make large areas accessible. A cord length of 50 feet or more is suggested.
6. The unit must not be plugged into telecommunications equipment frame outlets. Only plug vacuum cleaners into AC outlets that are on the walls not the equipment frames; this must be a separate AC circuit from the equipment.
7. The vacuum cleaner motor’s AC ground and any brush or wand ground must be isolated from each other.
8. An indicator should be provided to alert the operator when a filter change is needed.
9. EMI suppression circuits should be built into the motor assembly.
10. ESD dissipation should be built into the brush to reduce ESD, but with insulated hoses and exposed surfaces to prevent shorting of components by accidental contact. A clip or other means of fastening a grounding strap to the nozzle is desirable.
11. A list of vacuum cleaning precautions is on the next page.

 

Information is available from the Telcordia Disaster Prevention and Recovery Group on brushes that will facilitate vacuuming of internal switch surfaces without turning off power to the shelves. This requires an insulating hose and a special ESD brush. ESD brushes are now available for use in the cleaning of operational (powered) telecommunications equipment. These brushes are available through your local 3M electronics supplier. We can provide a loan of a small number of these brushes for emergency use during the cleaning procedures. Vacuum cleaners that we have seen used successfully on switching equipment include those made by 3M (commercial electronics suppliers); Hako Minuteman, (800) 323-9420; and Nilfisk of America, (800) NIL-FISK.

 

PRECAUTIONS FOR THE USAGE OF VACUUMS ON ELECTRONIC

EQUIPMENT SURFACES

 

WARNING – Certain precautions apply whenever one uses devices obtained from other than a telephone switch equipment manufacturer. This is particularly true of motor driven appliances.

 

1. Care must be exercised so that the ESD grounding points of the device, operator, and power source are compatible with the units being cleaned.
2. The operator, equipment being cleaned, and the device the operator is using on the equipment should all be connected to the same ground point. That is, the operator’s grounding wrist strap and a grounding strap placed on the vacuum cleaner brush should be as close to the tip as possible if no connection is provided on the brush itself. This configuration with standard brushes still does not allow vacuuming of live components. The ESD dissipating brushes should be used during the cleaning of internal surfaces on a powered frame.
3. Inadvertent bumping of the grounded, metallic parts of the motor housing into equipment at different ground points must be avoided. Insulating, plastic housings are preferred.
4. Many devices, specifically vacuum cleaners, have fans that will blow air out of the exhaust. This exhaust should not hit uncleaned surfaces where inadvertent secondary dirt sources could be generated.
5. The proper procedure for cleaning is to start at one corner or side of a particular electronic equipment area and work away from that point so that the secondary air source coming from the motor only hits previously cleaned surfaces.
6. Vacuuming of powered electronic equipment should NOT be attempted if the relative humidity of the space is below 25% RH.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Introduction

 

This procedure is intended for the cleaning of the two Lucent 5ESS digital switches (DS1 & DS2) located on the 7^th floor of 140 West Street and includes procedures for associated sub-floor space, cable racks, hardware and facility surfaces. This hardware and equipment space has been contaminated with particulate that includes asbestos fiber, wallboard and concrete dust, and common soils. Note that the 5ESS equipment contains sub-assemblies of circuit packs, connectors and cabling that also require a detailed level of cleaning.  To clean the 5ESS equipment, persons knowledgeable in the disassembly, functionality, trouble-shooting and powering requirements must be available during all phases of the 5ESS cleaning operation. Planning should include procedures for putting redundant equipment in simplex and de-powering line units and other systems. Details must also be provided to remove and restore customer service. Telcordia assumes that all cleaning will commence during maintenance window hours.  Telcordia Technologies’ Disaster Prevention and Recovery Group welcomes Verizon’s and their contractor(s) questions regarding specific details of this document. Please read the entire document before proceeding with cleaning operations. This is a dry cleaning process only. Do not use wet techniques, e.g., water or solvent wiping.

 

Equipment Needs

 

• Personal Protective Equipment (PPE) and personal training as specified for the 140 West Street Central Office by Verizon and all applicable Local, State and Federal laws.
• Electrostatic Field (ESF) meter. A suggested scale is 0 to 20 kV/inch.
• Static safe portable workstation. Equipped with HEPA filtered chamber, ESD dissipative grounded mats on table and floor.
• HEPA filtered vacuum cleaner. This must have an insulating hose section where it is connected to the ESD vacuum cleaner brush such that the ESD brush ground and the AC motor ground are not connected to the same electrical point. (See separate, attached vacuum cleaner specification.)
• ESD vacuum cleaner brush and wand. The brush and wand must be grounded to the equipment ground plane and be isolated from the AC ground plane.
• ESD wrist straps and controls for personnel.
• Extra circuit packs for all of the models and versions of circuit packs being cleaned.
• ESD dissipative circuit pack carrying case.
• If required, portable room humidifiers. Depending on the size of the office, outside temperature and relative humidity (RH), and rate of air exchange; multiple humidifiers may be required.
• Portable RH meter. This should be located at the work site where the pack vacuum cleaning is being done. This should not be the centralized location of the central office RH sensor.
• Electronic grade tack cloths.
• Compressed air or nitrogen.
• Extension cords with ground lead.
• Tool for pulling raised floor tiles.
• A portable fluorescent drop light and extension cord for each cleaning technician.
• Spare 5ESS circuit packs.
• 5ESS fan blockers or procedure for de-powering fans during cleaning.
• 5ESS frame filters for each frame.

General Considerations

 

Much of the particulate at the 140 West Street Central Office has been found to contain asbestos.   Personnel protective equipment (PPE) and safety training are required to perform this work and Telcordia strongly recommends that PPE and training be implemented per all Verizon and applicable Local, State and Federal requirements.

 

In conducting particulate cleaning, there is a risk of component damage from ESD (electrostatic discharge) when individual circuit packs are cleaned, but with proper precautions, cleaning can be accomplished successfully. When cleaning circuit packs, they should be carefully removed from the frames with minimal vibration and taken to an anti‑static work area (a bench with an appropriately grounded static dissipating surface or pad). Exercise precautions to ensure that particles do not contaminate other adjacent circuit packs, connectors, etc. It is intended that all circuit packs from a particular shelf that are permissible to be removed under Simplex operation be done at one time, and to expedite the cleaning process, the internal framework surfaces, the card cage and the circuit pack connector side of the backplane be cleaned simultaneously. Procedures for cleaning internal and external equipment surfaces, circuit packs, and backplane pin fields are provided in the following sections.

 

Subfloor Cleaning

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2.
   1. Cleaning should only be undertaken by trained workers familiar with operating in an asbestos contaminated environment that houses sensitive network electronics.
   2. When vacuuming equipment and facility surfaces:
      1. Vacuuming must be done in a manner to minimize the distribution of dust or debris onto or into network equipment.
      2. Workers should always point vacuum exhaust away from uncleaned surfaces.
      3. When full, vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit  personnel exposure or the re-contamination of equipment or facility surfaces.
      4. The space below the raised floor of the 7^th Floor 5ESS room should be cleaned first, proceeding from the area of greatest particulate deposition to the area of least particulate deposition (DS1 SM101 to DS2 SM01). Telcordia would suggest the HVAC filters be changed prior to the start of cleaning and changed again immediately after the cleaning procedure is completed.
      5. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets on the equipment frames.
      6. Using a HEPA vacuum that meets industry standard asbestos remediation guidelines and the Telcordia criteria, vacuum the top surface of the tile to be pulled.
      7. Pull the floor tile using the floor tile puller, then vacuum the remaining five sides of the floor tile.  Set aside.
      8. Place lighting into space below raised floor.
      9. Using the HEPA vacuum, vacuum all accessible adjacent tile underside surfaces, then vacuum all floor stanchion and framework surfaces.
      10. Vacuum all accessible areas of the subfloor surface.
      11. Inspect vacuumed subfloor area to verify completeness of cleaning, using portable lighting in the subfloor space.
      12. Using an electronic grade tack cloth, wipe down all subfloor surfaces. After wiping, discard tack cloth as asbestos waste into a yellow plastic bag labeled in accordance with asbestos waste management regulations.
      13. Re-install floor tile.
      14. Repeat Steps 6 through 13 until all subfloor surfaces are cleaned.
      15. After cleaning of the entire subfloor, carefully clean all horizontal and vertical facility surfaces above the 5ESS equipment as outlined below.

 

Cleaning of Horizontal and Vertical Facility and Upper Cable Structures

 

1. When vacuuming equipment and facility surfaces:
   1. Vacuuming must be done in a manner to minimize the distribution of dust or debris onto or into network equipment.
   2. Workers should always point vacuum exhaust away from uncleaned surfaces.
   3. When full, vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit the re-contamination of equipment or facility surfaces.
   4. Secure all elevated or suspended tools and equipment to prevent it from falling onto equipment below.
   5. Cleaning should proceed from higher surfaces to lower surfaces.
   6. Cleaning should proceed from the area of greatest particulate deposition to the area of least particulate deposition (DS1 SM101 to DS2 SM01).
   7. Where cleaning could result in the debris dropping onto or being entrained into network equipment, protect it with fire retardant plastic sheeting.
   8. Vacuum all horizontal facility surfaces (ceiling, walls, cable, cable trays, cable racks, lighting, unistrut, wall-mounted equipment, etc.) using a HEPA vacuum. In areas where cables are not tightly bundled, spread cables as much as possible to maximize cleaning efficiency.
   9. Following cleaning of a particular surface inspect the surface for area for remnant dust.  Re-vacuum or wipe with tack cloth as necessary, then inspect using portable lighting before proceeding to the next work area.

 

Cleaning of  Top and Front of 5ESS Equipment

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
   1. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
   2. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets located on the equipment frames.
   3. Ground the operator and ESD brush of the vacuum to the equipment ground plane.
   4. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)  Periodically check ES field, and shut down vacuum if ES level of 2.5 kV/in is exceeded.
   5. Vacuum the top cable trough of the 5ESS frame and any cables contained therein. Inspect and, if necessary, re-vacuum or wipe with tack cloth.
   6. From the top of the front external surface of the frame working downward, vacuum the exterior of the cabinet doors and framework.
   7. On frames with accordion-style doors:
      1. Flip up the blue cover above cabinet doors to access the fuse panel.  Vacuum all accessible fuse holder and framework surfaces.  Inspect and re-vacuum as necessary, then close the cover. 
      2. If necessary, remove the doors from the frame for cleaning of internal door surfaces.
      3. On frames with cabinet-style doors:
         1. Open the doors and clean the internal surfaces.
         2. Flip down the brass-colored fuse panel cover. Vacuum all accessible fuse holder and framework surfaces.  Inspect and re-vacuum as necessary, then close the cover. 
         3. Where present, lift the circuit pack identification strip at the top of each shelf and vacuum the underside of the strip.
         4. Vacuum the fronts of all circuit packs and any fuse holders at the top of the fame. Care must be taken to avoid changing any toggle equipment or push button settings on circuit packs. 
         5. Vacuum all accessible framework surfaces.
         6. Inspect cleaned front-of-frame surfaces, re-vacuuming or wiping if necessary.

 

5ESS Circuit Pack Removal and Shelf Cleaning

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. On the frame to be cleaned, review functionality and, where applicable, have a central office technician put equipment into simplex operation.  Discuss the location of redundant equipment within each frame with the central office technician, and remove only those circuit packs indicated by him or her to have been taken out of service, or that are otherwise ready for removal.
2. Record the location (SM, Shelf No., Slot No.) and model codes for each circuit pack to be removed in the top shelf of the frame.  Repeat for other shelves in the frame that have been put into simplex operation. It is critical that the exact pack removed from a card slot be re-installed in the same position after cleaning. 
3. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance from the brush head that is recommended for the particular meter used; this distance is typically  one to two inches..  See meter instructions. Periodically check ES field, and shut down vacuum if ES level of 2.5 kV/in is exceeded.
4. Install 5ESS fan blockers or de-power fan shelf per Lucent specification and Verizon instruction and remove frame filter and place in approved disposal bag.  When fans are blocked or de-powered, all front and rear frame doors must remain fully open.
5. Using ESD protection, unlatch and begin to pull a circuit pack from its slot, and place it in a static safe circuit pack carrier or bag.

WARNING ‑ When a pack is removed from the frame, contaminants on or around it will become airborne. It is important to maintain the vacuum in a position consistent with capturing the loose particulate. In addition, placing a piece of blotter paper under the pack to be removed will help minimize cross-contamination of other equipment surfaces.

1. Continue removing until one entire shelf of circuit packs is complete, or in the case of shelves in simplex, until all of the out-of-service circuit packs in the frame are removed.
2. Using static-safe carriers, move the circuit packs to a grounded workstation for thorough cleaning. To expedite the cleaning procedure, this operation should be done by dedicated individuals as outlined under the Circuit Pack Cleaning Procedure section of this document.
3. From the front of the emptied card slots, vacuum all accessible shelf internal framework and card cage surfaces, both vertical and horizontal.
4. Vacuum the front (internal side) of the backplane.  Carefully vacuum the connectors on the front of the backplane. Care must be taken to avoid any deposition of the vacuum brush fibers across pins or connectors should a fiber come loose from the brush. When a particular shelf is completed, verify that no remnant dust or bristles have been left behind.
5. If vacuuming does not remove all of the dust, the framework and circuit pack cage may be further cleaned by using a tack cloth. Do not use tack cloths on the internal connector surfaces of backplane connectors.
6. After cleaning at a separate workstation as outlined under the Circuit Pack Cleaning section of this document, replace circuit packs in the exact positions from which they were removed.
7. Have the central office technician return installed circuit packs to service and run diagnostics.
8. Have the central office technician put into simplex operations the next group of circuit packs.  Repeat Steps 2 through 12 until all of the circuit packs and shelves in the frame are cleaned. Note, some packs will not be placed in Simplex, e.g., grid packs, confer with Verizon technician(s) on removing these packs for cleaning.
9. Install new frame filter and remove fan blockers or re-power fan shelf.
10. Vacuum the floor under and around the frame and any remaining framework.
11. Proceed to rear of 5ESS frame cleaning below.

 

Cleaning of Rear of 5ESS Equipment

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
2. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
3. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets on the equipment frames.
4. Ground the operator and ESD brush of the vacuum to the equipment ground plane.
5. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)  Periodically check ES field, and shut down vacuum if ES level of 2.5 kV/in is exceeded.
6. From the top of the rear external surface of the frame working downward, vacuum the exterior of the cabinet doors and framework.
7. Open the cabinet doors and vacuum interior door surfaces.  If necessary, accordion-style doors should be removed from the frame for cleaning.
8. Starting at the top of the frame, begin to vacuum all cables, wires, connectors, and adjacent surfaces.  If possible, a sheet of blotter paper should be held under the area being vacuumed to catch falling debris. Wires and cables should be spread as much as possible to gain access for vacuuming. For vertical cables, it may be necessary to cut the lacing cord to access all areas. These cables must be re-laced following testing/cleaning in a manner consistent with industry practices. Care must be taken to avoid any deposition of the brush fibers across pins or connectors should a fiber come loose from the brush.
9. Vacuum all immediately accessible internal non-backplane framework surfaces.
10. Starting at the top shelf and proceeding down, carefully vacuum the rear of the backplane pin fields taking extreme care not to bend or deform the pins.
11. Periodically, check the level of the ES field at the brush to ensure that it is less than 2.5 kV/in.
12. After the cleaning of the back of the bank is complete, the backplane should be thoroughly inspected for remnant dust. Any dust should be re-vacuumed. Brush fibers should be carefully removed by hand.
13. If removed, re-install doors.
14. Vacuum the floor and any remaining framework.
15. Repeat Steps 4 through 14 for each frame.
16. Vacuum all remaining floor space within the equipment room including all work and terminal areas.
17. Proceed to cleaning the top and front of the next adjacent frame.

 

Circuit Pack Cleaning Procedure

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
   1. Adjust the relative humidity of the circuit pack cleaning space (or the room where the packs will be cleaned if it is another room) to over 25% RH and less than 45%. This RH should be measured at the location where the packs are being cleaned.
   2. Verify that there are multiple spare circuit packs of each model and version of circuit pack to be cleaned available on-site.
   3. Set up the filtered static-safe workstation, vacuum cleaner head, and personnel grounding system.  These three points should be connected to a common ground. Under no circumstances should you use the same ground as the AC vacuum cleaner or negative air machine motor.
   4. Using ESD protection, remove the packs to be cleaned from the shelf. Move the packs to the grounded, HEPA-filtered workstation. The person’s wrist strap performing the work and the ESD vacuum brush ground strap must be connected to the same ground point as the static-safe grounded workstation.
   5. Run HEPA vacuum cleaner for at least three minutes. The vacuum cleaner AC power should be from the building power, and should not be contained from the equipment frame.
   6. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)
   7. Vacuum the circuit pack on both sides. 
   8. Inspect the pack for cleanliness.  If dust of debris remains around hard-to access components, place circuit pack back in HEPA-filtered work chamber and blow off debris with compressed nitrogen or air (~15 psi), then re-vacuum the pack using the ESD vacuum brush.  Verify that no bristles from the vacuum have been caught on the pack components or connectors.
   9. In the HEPA-filtered work chamber, blow out the connector on the rear edge of the card
   10. Place the circuit pack in a cleaned static-safe carrying container.
   11. Vacuum the workstation cleaning surface.
   12. Proceed to the next pack and repeat Steps 5 through 12 of this procedure.
   13. When a shelf of packs is completed, and the framework cleaning has been accomplished for this shelf, re-insert the packs in their original shelf, ensuring that each circuit pack is returned to the same slot from which it was removed.

 

 

 

 

 

 HEPA VACUUM CLEANERS FOR CLEANING

ELECTRONIC EQUIPMENT

 

Shop vacuums, industrial vacuums, home and office vacuums, wet and dry vacuums, etc., that are designed for general use are not acceptable for telecommunications equipment applications. They are usually not efficient enough to handle submicron particle sizes, often generate ESD, often are not shielded for EMI, and may raise humidity levels.

 

To be used successfully on electronic equipment, the units should have the following characteristics:

1. The unit should have multilevel filtering with the final stage being a high efficiency particle arrestance (HEPA) filter, with a rating of at least 99.97% removal efficiency for particles down to 0.3 micrometers in diameter or larger.
2. Wet scrubber vacuum systems should not be used in central offices due to concerns over suddenly raising humidity levels, inefficiency of filtration, and potential spills.
3. Hoses and wands should be of sufficient length to allow cleaning of cable racks and under the base of electronic equipment; 25 feet is suggested. Hoses and any components, which may come into contact with electronic equipment, must be insulating.
4. Brush fiber lengths should be of the order of 5 cm. (2 inches) to allow cleaning action around and through cabling, wiring, channels, etc.
5. Power cords should be of sufficient length to make large areas accessible. A cord length of 50 feet or more is suggested.
6. The unit must not be plugged into telecommunications equipment frame outlets. Only plug vacuum cleaners into AC outlets that are on the walls not the equipment frames; this must be a separate AC circuit from the equipment.
7. The vacuum cleaner motor’s AC ground and any brush or wand ground must be isolated from each other.
8. An indicator should be provided to alert the operator when a filter change is needed.
9. EMI suppression circuits should be built into the motor assembly.
10. ESD dissipation should be built into the brush to reduce ESD, but with insulated hoses and exposed surfaces to prevent shorting of components by accidental contact. A clip or other means of fastening a grounding strap to the nozzle is desirable.
11. A list of vacuum cleaning precautions is on the next page.

 

Information is available from the Telcordia Disaster Prevention and Recovery Group on brushes that will facilitate vacuuming of internal switch surfaces without turning off power to the shelves. This requires an insulating hose and a special ESD brush. ESD brushes are now available for use in the cleaning of operational (powered) telecommunications equipment. These brushes are available through your local 3M electronics supplier. We can provide a loan of a small number of these brushes for emergency use during the cleaning procedures. Vacuum cleaners that we have seen used successfully on switching equipment include those made by 3M (commercial electronics suppliers); Hako Minuteman, (800) 323-9420; and Nilfisk of America, (800) NIL-FISK.

 

 

 

PRECAUTIONS FOR THE USAGE OF VACUUMS ON ELECTRONIC

EQUIPMENT SURFACES

 

WARNING – Certain precautions apply whenever one uses devices obtained from other than a telephone switch equipment manufacturer. This is particularly true of motor driven appliances.

 

1. Care must be exercised so that the ESD grounding points of the device, operator, and power source are compatible with the units being cleaned.
2. The operator, equipment being cleaned, and the device the operator is using on the equipment should all be connected to the same ground point. That is, the operator’s grounding wrist strap and a grounding strap placed on the vacuum cleaner brush should be as close to the tip as possible if no connection is provided on the brush itself. This configuration with standard brushes still does not allow vacuuming of live components. The ESD dissipating brushes should be used during the cleaning of internal surfaces on a powered frame.
3. Inadvertent bumping of the grounded, metallic parts of the motor housing into equipment at different ground points must be avoided. Insulating, plastic housings are preferred.
4. Many devices, specifically vacuum cleaners, have fans that will blow air out of the exhaust. This exhaust should not hit uncleaned surfaces where inadvertent secondary dirt sources could be generated.
5. The proper procedure for cleaning is to start at one corner or side of a particular electronic equipment area and work away from that point so that the secondary air source coming from the motor only hits previously cleaned surfaces.
6. Vacuuming of powered electronic equipment should NOT be attempted if the relative humidity of the space is below 25% RH.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Introduction

 

This procedure is intended for the cleaning of the Nortel DMS digital switching equipment located on the 7^th and 9^th floors of 140 West Street and includes procedures for associated sub-floor space, cable racks, hardware and facility surfaces. The 9^th floor DMS requires external cleaning as well as cleaning of internal surfaces in fan-cooled frames.  The 7^th floor DMS requires external cleaning only.

 

 Theses hardware and equipment spaces have been contaminated with particulate that includes asbestos fiber, wallboard and concrete dust, and common soils. Note that the DMS equipment contains sub-assemblies of circuit packs, connectors and cabling that also require a detailed level of cleaning.  To clean the DMS equipment, persons knowledgeable in the disassembly, functionality, trouble-shooting and powering requirements must be available during all phases of the DMS cleaning operation. Planning should include procedures for putting redundant equipment in simplex and de-powering line units and other systems. Details must also be provided to remove and restore customer service. Telcordia assumes that all cleaning will commence during maintenance window hours.  Telcordia Technologies’ Disaster Prevention and Recovery Group welcomes Verizon’s and their contractor(s) questions regarding specific details of this document. Please read the entire document before proceeding with cleaning operations. This is a dry cleaning process only. Do not use wet techniques, e.g., water or solvent wiping.

 

Equipment Needs

 

• Personal Protective Equipment (PPE) and personal training as specified for the 140 West Street Central Office by Verizon and all applicable Local, State and Federal laws.
• Electrostatic Field (ESF) meter. A suggested scale is 0 to 20 kV/inch.
• Static safe portable workstation. Equipped with HEPA filtered chamber, ESD dissipative grounded mats on table and floor.
• HEPA filtered vacuum cleaner. This must have an insulating hose section where it is connected to the ESD vacuum cleaner brush such that the ESD brush ground and the AC motor ground are not connected to the same electrical point. (See separate, attached vacuum cleaner specification.)
• ESD vacuum cleaner brush and wand. The brush and wand must be grounded to the equipment ground plane and be isolated from the AC ground plane.
• ESD wrist straps and controls for personnel.
• Extra circuit packs for all of the models and versions of circuit packs being cleaned.
• ESD dissipative circuit pack carrying case.
• If required, portable room humidifiers. Depending on the size of the office, outside temperature and relative humidity (RH), and rate of air exchange; multiple humidifiers may be required.
• Portable RH meter. This should be located at the work site where the pack vacuum cleaning is being done. This should not be the centralized location of the central office RH sensor.
• Electronic grade tack cloths.
• Compressed air or nitrogen.
• Extension cords with ground lead.
• Tool for pulling raised floor tiles.
• A portable fluorescent drop light and extension cord for each cleaning technician.
• Spare 5ESS circuit packs.
• Fan blockers or procedure for de-powering fans during cleaning.
• DMS frame filters for each fan-cooled frame
• Nortel Line Card Removal Tool.
• Non-metallic ladders.
• Two HEPA-filtered negative air machines equipped with 0.3 micron filtration and a minimum of 25 feet of clean, insulated, large diameter intake duct, and insulated duct for routing exhaust stream.

 

General Considerations

 

Much of the particulate at the 140 West Street Central Office has been found to contain asbestos.   Personnel protective equipment (PPE) and safety training are required to perform this work and Telcordia strongly recommends that PPE and training be implemented per all Verizon and applicable Local, State and Federal requirements.

 

In conducting particulate cleaning on the 9^th floor DMS, there is a risk of component damage from ESD (electrostatic discharge) when individual circuit packs are cleaned, but with proper precautions, cleaning can be accomplished successfully. When cleaning circuit packs, they should be carefully removed from the frames with minimal vibration and taken to an anti‑static work area (a bench with an appropriately grounded static dissipating surface or pad). Exercise precautions to ensure that particles do not contaminate other adjacent circuit packs, connectors, etc. It is intended that all circuit packs from a particular shelf that are permissible to be removed under Simplex operation be done at one time, and to expedite the cleaning process, the internal framework surfaces, the card cage and the circuit pack connector side of the backplane be cleaned simultaneously. Procedures for cleaning internal and external equipment surfaces, circuit packs, and backplane pin fields are provided in the following sections.

 

Subfloor Cleaning

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2.
2. Cleaning should only be undertaken by trained workers familiar with operating in an asbestos contaminated environment that houses sensitive network electronics.
3. When vacuuming equipment and facility surfaces:
   1. Vacuuming must be done in a manner to minimize the distribution of dust or debris onto or into network equipment.
   2. Workers should always point vacuum exhaust away from uncleaned surfaces.
   3. When full, vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit  personnel exposure or the re-contamination of equipment or facility surfaces.
   4. The space below the raised floor should be cleaned first, proceeding from the area of greatest particulate deposition to the area of least particulate deposition.
   5. Telcordia would suggest the HVAC filters be changed prior to the start of cleaning and changed again immediately after the cleaning procedure is completed.
   6. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets on the equipment frames.
   7. Using a HEPA vacuum that meets industry standard asbestos remediation guidelines and the Telcordia criteria, vacuum the top surface of the tile to be pulled.
   8. Pull the floor tile using the floor tile puller, then vacuum the remaining five sides of the floor tile.  Set aside.
   9. Place lighting into space below raised floor.
   10. Using the HEPA vacuum, vacuum all accessible adjacent tile underside surfaces, then vacuum all floor stanchion and framework surfaces.
   11. Vacuum all accessible areas of the subfloor surface.
   12. Inspect vacuumed subfloor area to verify completeness of cleaning, using portable lighting in the subfloor space.
   13. Using an electronic grade tack cloth, wipe down all subfloor surfaces. After wiping, discard tack cloth as asbestos waste into a yellow plastic bag labeled in accordance with asbestos waste management regulations.
   14. Re-install floor tile.
   15. Repeat Steps 6 through 13 until all subfloor surfaces are cleaned.
   16. After cleaning of the entire subfloor, carefully clean all horizontal and vertical facility surfaces above the 5ESS equipment as outlined below.

 

Cleaning of Horizontal and Vertical Facility and Upper Cable Structures

 

1. When vacuuming equipment and facility surfaces:
   1. Vacuuming must be done in a manner to minimize the distribution of dust or debris onto or into network equipment.
   2. Workers should always point vacuum exhaust away from uncleaned surfaces.
   3. When full, vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit the re-contamination of equipment or facility surfaces.
   4. Secure all elevated or suspended tools and equipment to prevent it from falling onto equipment below.
   5. Cleaning should proceed from higher surfaces to lower surfaces.
   6. Cleaning should proceed from the area of greatest particulate deposition to the area of least particulate deposition.
   7. Where cleaning could result in the debris dropping onto or being entrained into network equipment, protect it with fire retardant plastic sheeting.
   8. Vacuum all horizontal facility surfaces (ceiling, walls, cable, cable trays, cable racks, lighting, unistrut, wall-mounted equipment, etc.) using a HEPA vacuum. In areas where cables are not tightly bundled, spread cables as much as possible to maximize cleaning efficiency.
   9. Following cleaning of a particular surface inspect the surface for area for remnant dust.  Re-vacuum or wipe with tack cloth as necessary, then inspect using portable lighting before proceeding to the next work area.

 

Cleaning of  Top and Front of DMS Equipment

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
   1. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
   2. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets located on the equipment frames.
   3. Ground the operator and ESD brush of the vacuum to the equipment ground plane.
   4. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)  Periodically check ES field, and shut down vacuum if ES level of 2.5 kV/in is exceeded.
   5. Vacuum the top surfaces of the 5ESS frame and any adjacent cables. Inspect and, if necessary, re-vacuum or wipe with tack cloth.
   6. From the top of the front external surface of the frame working downward, vacuum the exterior of the cabinet doors and framework.
   7. Remove slotted metal panels at the tops of frames by unscrewing hex sheet metal screws, then vacuum panel surfaces and all accessible framework, cable, and component surfaces.
      1. On frames with cabinet-style doors:
      2. Vacuum exterior door surfaces
      3. Remove doors by pulling hinge pins.
      4. Vacuum internal door surfaces.
      5. Vacuum all accessible equipment surfaces. Note: Care must be taken to avoid changing any toggle equipment or push button settings on circuit packs. 
      6. On frames without doors:
      7. Vacuum the fronts of all circuit packs, drawers, intake gills, fuse holders, etc.  Note: Care must be taken to avoid changing any toggle equipment or push button settings on circuit packs. 
      8. Unscrew and swing out grill with fuses, located between equipment shelves, and vacuum the top surface of the heat shield.
      9. Inspect cleaned front-of-frame surfaces, re-vacuuming or wiping if necessary.
      10. Replace grills, doors, and panels.

 

Circuit Pack Removal and Shelf Cleaning for 9th Floor DMS Fan-Cooled Frames

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. On the frame to be cleaned, review functionality and, where applicable, have a central office technician put equipment into simplex operation.  Discuss the location of redundant equipment within each frame with the central office technician, and remove only those circuit packs indicated by him or her to have been taken out of service, or that are otherwise ready for removal.
2. Record the location (Frame, Shelf No., Slot No.) and model codes for each circuit pack to be removed in the top shelf of the frame.  Repeat for other shelves in the frame that have been put into simplex operation. Note:  It is critical that the exact pack removed from a card slot be re-installed in the same position after cleaning.    
3. Locate one negative air machine equipped with 0.3 micron filtration at each end of the line up to be cleaned.  Ensure that the exhaust stream is directed away from uncleaned areas.
4. Obtain power for the negative air machine from house power, not outlets on the equipment frames.
5. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
6. Suspend one intake duct adjacent to each side of the frame shelf to be cleaned.
7. Start the negative air machine.
8. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance from the brush head that is recommended for the particular meter used; this distance is typically  one to two inches..  See meter instructions. Periodically check ES field, and shut down vacuum if ES level of 2.5 kV/in is exceeded.
9. Install fan blockers or de-power fan shelf per Nortel specification and Verizon instruction and remove frame filter and place in approved disposal bag.  When fans are blocked or de-powered, all front and rear frame doors must remain fully open.  Note:  Fans should not be left blocked or de-powered in excess of 30 minutes. 
10. Using ESD protection, unlatch and begin to pull a circuit pack from its slot, and place it in a static safe circuit pack carrier or bag.

WARNING ‑ When a pack is removed from the frame, contaminants on or around it will become airborne. It is important to maintain the vacuum in a position consistent with capturing the loose particulate. In addition, placing a piece of blotter paper under the pack to be removed will help minimize cross-contamination of other equipment surfaces.

1. Continue removing until one entire shelf of circuit packs is complete.
2. Using static-safe carriers, move the circuit packs to a grounded workstation for thorough cleaning. To expedite the cleaning procedure, this operation should be done by dedicated individuals as outlined under the Circuit Pack Cleaning Procedure section of this document.
3. From the front of the emptied card slots, vacuum all accessible shelf internal framework and card cage surfaces, both vertical and horizontal.
4. Vacuum the front (internal side) of the backplane.  Carefully vacuum the connectors on the front of the backplane. Care must be taken to avoid any deposition of the vacuum brush fibers across pins or connectors should a fiber come loose from the brush. When a particular shelf is completed, verify that no remnant dust or bristles have been left behind.
5. If vacuuming does not remove all of the dust, the framework and circuit pack cage may be further cleaned by using a tack cloth. Do not use tack cloths on the internal connector surfaces of backplane connectors.
6. After cleaning at a separate workstation as outlined under the Circuit Pack Cleaning section of this document, replace circuit packs in the exact positions from which they were removed.
7. Have the central office technician return installed circuit packs to service and run diagnostics.
8. Reposition the air ducts of the negative air machine to the next lower shelf.
9. Have the central office technician put into simplex operations the next group of circuit packs.  Repeat Steps 2 through 18 until all of the circuit packs and shelves in the frame are cleaned. Note, some packs will not be placed in Simplex, e.g., grid packs, confer with Verizon technician(s) on removing these packs for cleaning.
10. After cleaning all shelves in the frame, install new frame filter and remove fan blockers or re-power fan shelf.
11. Vacuum the floor under and around the frame and any remaining framework.
12. Proceed to rear of DMS frame cleaning below.

 

Cleaning of Rear of DMS Equipment

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
2. Work is to proceed from the end of an aisle toward the center of the equipment lineup.
3. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets on the equipment frames.
4. Ground the operator and ESD brush of the vacuum to the equipment ground plane.
5. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)  Periodically check ES field, and shut down vacuum if ES level of 2.5 kV/in is exceeded.
6. From the top of the rear external surface of the frame working downward, vacuum the exterior of the cabinet doors and framework.
7. Open the cabinet doors and vacuum interior door surfaces.  If necessary, remove doors to facilitate access.
8. Starting at the top of the frame, begin to vacuum all cables, wires, connectors, and adjacent surfaces.  If possible, a sheet of blotter paper should be held under the area being vacuumed to catch falling debris. Wires and cables should be spread as much as possible to gain access for vacuuming. For vertical cables, it may be necessary to cut the lacing cord to access all areas. These cables must be re-laced following testing/cleaning in a manner consistent with industry practices. Care must be taken to avoid any deposition of the brush fibers across pins or connectors should a fiber come loose from the brush.
9. Vacuum all immediately accessible internal non-backplane framework surfaces.
10. Starting at the top shelf and proceeding down, carefully vacuum the rear of the backplane pin fields taking extreme care not to bend or deform the pins.
11. Periodically, check the level of the ES field at the brush to ensure that it is less than 2.5 kV/in.
12. After the cleaning of the back is complete, the backplane should be thoroughly inspected for remnant dust. Any dust should be re-vacuumed. Brush fibers should be carefully removed by hand.
13. If removed, re-install doors.
14. Vacuum the floor and any remaining framework.
15. Repeat Steps 4 through 14 for each frame.
16. Proceed to cleaning the top and front of the next adjacent frame.
17. When all frames are complete, vacuum all remaining floor space within the equipment room including all work and terminal areas.

 

Circuit Pack Cleaning Procedure

 

Note that the cleaning of electronic equipment in this facility requires that the space relative humidity remain between 25% and 45%.

 

1. Read the complete procedure through and understand the procedure before beginning with Step #2. 
2. Adjust the relative humidity of the circuit pack cleaning space (or the room where the packs will be cleaned if it is another room) to over 25% RH and less than 45%. This RH should be measured at the location where the packs are being cleaned.
3. Verify that there are multiple spare circuit packs of each model and version of circuit pack to be cleaned available on-site.
4. Set up the filtered static-safe workstation, vacuum cleaner head, and personnel grounding system.  These three points should be connected to a common ground. Under no circumstances should you use the same ground as the AC vacuum cleaner or negative air machine motor.
5. Using ESD protection, remove the packs to be cleaned from the shelf. Move the packs to the grounded, HEPA-filtered workstation. The person’s wrist strap performing the work and the ESD vacuum brush ground strap must be connected to the same ground point as the static-safe grounded workstation.
6. Run HEPA vacuum cleaner for at least three minutes. The vacuum cleaner AC power should be from the building power, and should not be contained from the equipment frame.
7. Verify that the ES field at the vacuum cleaner brush is less than 2.5 kV/in after the vacuum has been running for at least three minutes. (kV/in is the electrostatic field strength in kilovolts at a distance of inches from the brush head; this distance is nominally one inch and depends on which meter is being used.  See meter instructions.)
8. Vacuum the circuit pack on both sides. 
9. Inspect the pack for cleanliness.  If dust of debris remains around hard-to access components, place circuit pack back in HEPA-filtered work chamber and blow off debris with compressed nitrogen or air (~15 psi), then re-vacuum the pack using the ESD vacuum brush.  Verify that no bristles from the vacuum have been caught on the pack components or connectors.
10. In the HEPA-filtered work chamber, blow out the connector on the rear edge of the card
11. Place the circuit pack in a cleaned static-safe carrying container.
12. Vacuum the workstation cleaning surface.
13. Proceed to the next pack and repeat Steps 5 through 12 of this procedure.
14. When a shelf of packs is completed, and the framework cleaning has been accomplished for this shelf, re-insert the packs in their original shelf, ensuring that each circuit pack is returned to the same slot from which it was removed.

 

 

HEPA VACUUM CLEANERS FOR CLEANING

ELECTRONIC EQUIPMENT

 

Shop vacuums, industrial vacuums, home and office vacuums, wet and dry vacuums, etc., that are designed for general use are not acceptable for telecommunications equipment applications. They are usually not efficient enough to handle submicron particle sizes, often generate ESD, often are not shielded for EMI, and may raise humidity levels.

 

To be used successfully on electronic equipment, the units should have the following characteristics:

1. The unit should have multilevel filtering with the final stage being a high efficiency particle arrestance (HEPA) filter, with a rating of at least 99.97% removal efficiency for particles down to 0.3 micrometers in diameter or larger.
2. Wet scrubber vacuum systems should not be used in central offices due to concerns over suddenly raising humidity levels, inefficiency of filtration, and potential spills.
3. Hoses and wands should be of sufficient length to allow cleaning of cable racks and under the base of electronic equipment; 25 feet is suggested. Hoses and any components, which may come into contact with electronic equipment, must be insulating.
4. Brush fiber lengths should be of the order of 5 cm. (2 inches) to allow cleaning action around and through cabling, wiring, channels, etc.
5. Power cords should be of sufficient length to make large areas accessible. A cord length of 50 feet or more is suggested.
6. The unit must not be plugged into telecommunications equipment frame outlets. Only plug vacuum cleaners into AC outlets that are on the walls not the equipment frames; this must be a separate AC circuit from the equipment.
7. The vacuum cleaner motor’s AC ground and any brush or wand ground must be isolated from each other.
8. An indicator should be provided to alert the operator when a filter change is needed.
9. EMI suppression circuits should be built into the motor assembly.
10. ESD dissipation should be built into the brush to reduce ESD, but with insulated hoses and exposed surfaces to prevent shorting of components by accidental contact. A clip or other means of fastening a grounding strap to the nozzle is desirable.
11. A list of vacuum cleaning precautions is on the next page.

 

Information is available from the Telcordia Disaster Prevention and Recovery Group on brushes that will facilitate vacuuming of internal switch surfaces without turning off power to the shelves. This requires an insulating hose and a special ESD brush. ESD brushes are now available for use in the cleaning of operational (powered) telecommunications equipment. These brushes are available through your local 3M electronics supplier. We can provide a loan of a small number of these brushes for emergency use during the cleaning procedures. Vacuum cleaners that we have seen used successfully on switching equipment include those made by 3M (commercial electronics suppliers); Hako Minuteman, (800) 323-9420; and Nilfisk of America, (800) NIL-FISK.

 

 

 

 

PRECAUTIONS FOR THE USAGE OF VACUUMS ON ELECTRONIC

EQUIPMENT SURFACES

 

WARNING – Certain precautions apply whenever one uses devices obtained from other than a telephone switch equipment manufacturer. This is particularly true of motor driven appliances.

 

1. Care must be exercised so that the ESD grounding points of the device, operator, and power source are compatible with the units being cleaned.
2. The operator, equipment being cleaned, and the device the operator is using on the equipment should all be connected to the same ground point. That is, the operator’s grounding wrist strap and a grounding strap placed on the vacuum cleaner brush should be as close to the tip as possible if no connection is provided on the brush itself. This configuration with standard brushes still does not allow vacuuming of live components. The ESD dissipating brushes should be used during the cleaning of internal surfaces on a powered frame.
3. Inadvertent bumping of the grounded, metallic parts of the motor housing into equipment at different ground points must be avoided. Insulating, plastic housings are preferred.
4. Many devices, specifically vacuum cleaners, have fans that will blow air out of the exhaust. This exhaust should not hit uncleaned surfaces where inadvertent secondary dirt sources could be generated.
5. The proper procedure for cleaning is to start at one corner or side of a particular electronic equipment area and work away from that point so that the secondary air source coming from the motor only hits previously cleaned surfaces.
6. Vacuuming of powered electronic equipment should NOT be attempted if the relative humidity of the space is below 25% RH.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Introduction

 

This procedure is intended for the cleaning of main distributing frames located within Verizon’s office at 140 West Street, Manhattan. Hardware and equipment spaces within the office have been contaminated with particulate that includes asbestos fiber, wallboard and concrete dust, and common soils. Only persons trained in working around electronic equipment and in working in asbestos-contaminated environments should undertake this cleaning. Telcordia assumes that all cleaning will commence during maintenance window hours.  Telcordia Technologies’ Disaster Prevention and Recovery Group welcomes Verizon’s and their contractor(s) questions regarding specific details of this document. Please read the entire document before proceeding with cleaning operations. This is a dry cleaning process only. Do not use wet techniques, e.g., water or solvent wiping.

 

Equipment Needs

 

• Personal Protective Equipment (PPE) and personal training as specified for the 140 West Street Central Office by Verizon and all applicable Local, State and Federal laws.
• HEPA filtered vacuum cleaner. This must have an insulating hose section where it is connected to the ESD vacuum cleaner brush such that the ESD brush ground and the AC motor ground are not connected to the same electrical point. (See separate, attached vacuum cleaner specification.)
• Insulated vacuum cleaner brush and wand attachments. The brush and wand must be grounded to the equipment frame ground plane and be isolated from the AC ground plane.
• ESD wrist straps and controls for personnel.
• Electronic grade tack cloths.
• A portable fluorescent droplight and extension cord for each cleaning technician.
• Fire retardant plastic sheeting.
• Ladders.

 

General Considerations

 

Particulate at the 140 West Street Central Office has been found to contain asbestos in many areas. Personnel protective equipment (PPE) and safety training are required to perform this work and Telcordia strongly recommends that PPE and training be implemented per all Verizon and applicable Local, State and Federal requirements.

 

Frame Room Facility Cleaning

 

1. When vacuuming frame and facility surfaces:
   1. Vacuuming must be done in a manner to minimize the distribution of dust or debris onto or into other frame and facility network elements.
   2. Workers should always point vacuum exhaust away from uncleaned surfaces.
   3. When full, vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit the re-contamination of equipment or facility surfaces.
   4. Secure all elevated or suspended tools and equipment to prevent it from falling onto equipment below.
   5. Cleaning should proceed from higher surfaces to lower surfaces.
   6. Where cleaning could result in the debris dropping onto or being entrained into equipment, protect it with fire retardant plastic sheeting.
   7. Vacuum all horizontal facility surfaces (ceiling, walls, cable and cable trays, lighting, unistrut, wall-mounted equipment, etc.) using a HEPA vacuum. In areas where cables are not tightly bundled, spread cables as much as possible and vacuum with brush or wand  to maximize cleaning efficiency.
   8. Following cleaning of a particular surface, inspect the surface for area for remnant dust.  Re-vacuum or wipe with tack cloth as necessary, then inspect using portable lighting before proceeding to the next work area.

 

Main Distributing Frame Cleaning

 

1. Work is to proceed from one end of the frame, and in the case of tiered frames, start at the top.
2. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained only from house ac circuits, not ac outlets located on network equipment frames.
3. Ground the operator and ESD brush of the vacuum to frame.  Do not attach personnel or equipment ESD grounding devices to pins on the vertical or horizontal blocks.
4. Vacuum the top of the frame and associated pair wires, spreading wires, where feasible, to allow for vacuuming in between. Exercise caution not to vacuum up in frame piece wire connectors.
5. Vacuum all vertical and horizontal surfaces, including the horizontal and vertical blocks.  Exercise car in vacuuming pin fields to ensure that no pins are bent or wire-wrap connections broken. Where equipped, open hinged blocks and vacuum behind surfaces.
6. Inspect cleaned surfaces, re-vacuum or wipe with tack cloth, if necessary.

 

HEPA VACUUM CLEANERS FOR CLEANING

ELECTRONIC EQUIPMENT

 

Shop vacuums, industrial vacuums, home and office vacuums, wet and dry vacuums, etc., that are designed for general use are not acceptable for telecommunications equipment applications. They are usually not efficient enough to handle submicron particle sizes, often generate ESD, often are not shielded for EMI, and may raise humidity levels.

 

To be used successfully on electronic equipment, the units should have the following characteristics:

1. The unit should have multilevel filtering with the final stage being a high efficiency particle arrestance (HEPA) filter, with a rating of at least 99.97% removal efficiency for particles down to 0.3 micrometers in diameter or larger.
2. Wet scrubber vacuum systems should not be used in central offices due to concerns over suddenly raising humidity levels, inefficiency of filtration, and potential spills.
3. Hoses and wands should be of sufficient length to allow cleaning of cable racks and under the base of electronic equipment; 25 feet is suggested. Hoses and any components, which may come into contact with electronic equipment, must be insulating.
4. Brush fiber lengths should be of the order of 5 cm. (2 inches) to allow cleaning action around and through cabling, wiring, channels, etc.
5. Power cords should be of sufficient length to make large areas accessible. A cord length of 50 feet or more is suggested.
6. The unit must not be plugged into telecommunications equipment frame outlets. Only plug vacuum cleaners into AC outlets that are on the walls not the equipment frames; this must be a separate AC circuit from the equipment.
7. The vacuum cleaner motor’s AC ground and any brush or wand ground must be isolated from each other.
8. An indicator should be provided to alert the operator when a filter change is needed.
9. EMI suppression circuits should be built into the motor assembly.
10. ESD dissipation should be built into the brush to reduce ESD, but with insulated hoses and exposed surfaces to prevent shorting of components by accidental contact. A clip or other means of fastening a grounding strap to the nozzle is desirable.
11. A list of vacuum cleaning precautions is on the next page.

 

Information is available from the Telcordia Disaster Prevention and Recovery Group on brushes that will facilitate vacuuming of internal switch surfaces without turning off power to the shelves. This requires an insulating hose and a special ESD brush. ESD brushes are now available for use in the cleaning of operational (powered) telecommunications equipment. These brushes are available through your local 3M electronics supplier. We can provide a loan of a small number of these brushes for emergency use during the cleaning procedures. Vacuum cleaners that we have seen used successfully on switching equipment include those made by 3M (commercial electronics suppliers); Hako Minuteman, (800) 323-9420; and Nilfisk of America, (800) NIL-FISK.

 

 

 

 

PRECAUTIONS FOR THE USAGE OF VACUUMS ON ELECTRONIC

EQUIPMENT SURFACES

 

WARNING – Certain precautions apply whenever one uses devices obtained from other than a telephone switch equipment manufacturer. This is particularly true of motor driven appliances.

 

1. Care must be exercised so that the ESD grounding points of the device, operator, and power source are compatible with the units being cleaned.
2. The operator, equipment being cleaned, and the device the operator is using on the equipment should all be connected to the same ground point. That is, the operator’s grounding wrist strap and a grounding strap placed on the vacuum cleaner brush should be as close to the tip as possible if no connection is provided on the brush itself. This configuration with standard brushes still does not allow vacuuming of live components. The ESD dissipating brushes should be used during the cleaning of internal surfaces on a powered frame.
3. Inadvertent bumping of the grounded, metallic parts of the motor housing into equipment at different ground points must be avoided. Insulating, plastic housings are preferred.
4. Many devices, specifically vacuum cleaners, have fans that will blow air out of the exhaust. This exhaust should not hit uncleaned surfaces where inadvertent secondary dirt sources could be generated.
5. The proper procedure for cleaning is to start at one corner or side of a particular electronic equipment area and work away from that point so that the secondary air source coming from the motor only hits previously cleaned surfaces.
6. Vacuuming of powered electronic equipment should NOT be attempted if the relative humidity of the space is below 25% RH.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Introduction

 

This procedure is intended for the external cleaning of telecommunications power equipment located within Verizon’s Office at 140 West Street, Manhattan. Hardware and equipment spaces within the office have been contaminated with particulate that includes asbestos fiber, wallboard and concrete dust, and common soils. Only persons trained in working with dc power plant equipment and in working in asbestos-contaminated environments should undertake this cleaning. To clean the power equipment, persons knowledgeable in the disassembly, functionality, trouble-shooting and powering requirements must be available during all phases of the cleaning operation. Telcordia assumes that all cleaning will commence during maintenance window hours. Telcordia Technologies’ Disaster Prevention and Recovery Group welcomes Verizon’s and their contractor(s) questions regarding specific details of this document. This procedure is intended to be performed only on accessible external power equipment surfaces. Please read the entire document before proceeding with cleaning operations.

 

Equipment Needs for Power Equipment

 

• Personal Protective Equipment (PPE) and personal training as specified for the 140 West Street Office by Verizon and all applicable Local, State and Federal laws.
• HEPA filtered vacuum cleaner. This must have an insulating hose section where it is connected to the ESD vacuum cleaner brush such that the ESD brush ground and the AC motor ground are not connected to the same electrical point. (See separate, attached vacuum cleaner specification.)
• Insulated vacuum cleaner brush and wand attachments. The brush and wand must be grounded to the equipment ground plane and be isolated from the AC ground plane.
• ESD wrist straps and controls for personnel.
• Electronic grade tack cloths.
• A portable fluorescent droplight and extension cord for each cleaning technician.
• Fire retardant plastic sheeting.
• Sheets of Masoniteâ and/or rubber mats to isolate exposed energized bus or battery connection straps
• Wooden or fiberglass ladders.

 

Equipment Needs for Batteries and Stands

 

• Personal Protective Equipment (PPE) and personal training as specified above.
• ESD wrist straps and controls for personnel.
• Electronic grade tack cloths or non-shedding lint cloths moistened with deionized or distilled water.
• A portable fluorescent droplight and extension cord for each cleaning technician.
• Fire retardant plastic sheeting.
• Sheets of Masoniteâ and/or rubber mats to isolate exposed energized bus or battery connection straps
• Wooden or fiberglass ladders.

 

 

General Considerations

 

Particulate at the 140 West Street Office has been found to contain asbestos in many areas. Personnel protective equipment (PPE) and safety training are required to perform this work and Telcordia strongly recommends that PPE and training be implemented per all Verizon and applicable Local, State and Federal requirements.

 

Power Room Facility Cleaning

 

1. When vacuuming equipment and facility surfaces:
   1. Vacuuming must be done in a manner to minimize the distribution of dust or debris onto or into network equipment.
   2. Workers should always point vacuum exhaust away from uncleaned surfaces.
   3. When full, vacuums should be emptied in a manner consistent with asbestos abatement projects, i.e., in a manner that does not permit the re-contamination of equipment or facility surfaces.
   4. Secure all elevated or suspended tools and equipment to prevent it from falling onto equipment below.
   5. Cleaning should proceed from higher surfaces to lower surfaces.
   6. Where personnel or tools could inadvertently contact or drop tools onto energized bus, battery connection straps, battery terminals, or other exposed, energized conductors, insulate same with Masoniteâ and/or rubber mats according to Verizon directive.
   7. Where cleaning could result in the debris dropping onto or being entrained into equipment, protect it with fire retardant plastic sheeting.
   8. Vacuum all horizontal facility surfaces (ceiling, walls, cable, cable trays, cable racks, lighting, unistrut, wall-mounted equipment, etc.) using a HEPA vacuum. In areas where cables are not tightly bundled, spread cables as much as possible to maximize cleaning efficiency.
   9. Following cleaning of a particular surface, inspect the surface for area for remnant dust.  Re-vacuum or wipe with tack cloth as necessary, then inspect using portable lighting before proceeding to the next work area.

 

Power Room Equipment Cleaning

 

1. Near the area to be cleaned, plug in the HEPA vacuum and lighting.  Power should be obtained from house ac circuits, not ac outlets located on the equipment frames.
2. Ground the operator and ESD brush of the vacuum to the equipment ground plane. 
3. Where personnel or tools could inadvertently contact or drop tools onto energized bus, battery connection straps, battery terminals, or other exposed, energized conductors, insulate same with Masoniteâ and/or rubber mats according to Verizon directive.
4. Vacuum the top of the rectifier cabinets, PDF’s, AC distribution and breaker cabinets, etc., and associated cables and chassis. From the top of the front external surface of the cabinet or frame, work downward, vacuuming the exterior of the cabinet doors and framework.
5. For rack-mounted equipment:  Vacuum all accessible vertical and horizontal surfaces.
6. Inspect cleaned surfaces, re-vacuuming or wiping with tack cloth, if necessary.

 

Battery Cleaning

 

1. Ensure that all safety vents are in place and electrolyte levels are at recommended height.
2. Ensure that the ESD wrist strap of the person performing the cleaning is attached to a ground point on the battery stand before the commencement of work and all precautions while working around batteries are followed to minimize sparks.
3. Using electronic grade tack cloth or a non-shedding lint cloth moistened with deionized or distilled water, wipe down battery and battery stand surfaces.
4. Inspect cleaned surfaces, re-wipe, if necessary.

 

HEPA VACUUM CLEANERS FOR CLEANING

ELECTRONIC EQUIPMENT

 

Shop vacuums, industrial vacuums, home and office vacuums, wet and dry vacuums, etc., that are designed for general use are not acceptable for telecommunications equipment applications. They are usually not efficient enough to handle submicron particle sizes, often generate ESD, often are not shielded for EMI, and may raise humidity levels.

 

To be used successfully on electronic equipment, the units should have the following characteristics:

1. The unit should have multilevel filtering with the final stage being a high efficiency particle arrestance (HEPA) filter, with a rating of at least 99.97% removal efficiency for particles down to 0.3 micrometers in diameter or larger.
2. Wet scrubber vacuum systems should not be used in central offices due to concerns over suddenly raising humidity levels, inefficiency of filtration, and potential spills.
3. Hoses and wands should be of sufficient length to allow cleaning of cable racks and under the base of electronic equipment; 25 feet is suggested. Hoses and any components, which may come into contact with electronic equipment, must be insulating.
4. Brush fiber lengths should be of the order of 5 cm. (2 inches) to allow cleaning action around and through cabling, wiring, channels, etc.
5. Power cords should be of sufficient length to make large areas accessible. A cord length of 50 feet or more is suggested.
6. The unit must not be plugged into telecommunications equipment frame outlets. Only plug vacuum cleaners into AC outlets that are on the walls not the equipment frames; this must be a separate AC circuit from the equipment.
7. The vacuum cleaner motor’s AC ground and any brush or wand ground must be isolated from each other.
8. An indicator should be provided to alert the operator when a filter change is needed.
9. EMI suppression circuits should be built into the motor assembly.
10. ESD dissipation should be built into the brush to reduce ESD, but with insulated hoses and exposed surfaces to prevent shorting of components by accidental contact. A clip or other means of fastening a grounding strap to the nozzle is desirable.
11. A list of vacuum cleaning precautions is on the next page.

 

Information is available from the Telcordia Disaster Prevention and Recovery Group on brushes that will facilitate vacuuming of internal switch surfaces without turning off power to the shelves. This requires an insulating hose and a special ESD brush. ESD brushes are now available for use in the cleaning of operational (powered) telecommunications equipment. These brushes are available through your local 3M electronics supplier. We can provide a loan of a small number of these brushes for emergency use during the cleaning procedures. Vacuum cleaners that we have seen used successfully on switching equipment include those made by 3M (commercial electronics suppliers); Hako Minuteman, (800) 323-9420; and Nilfisk of America, (800) NIL-FISK.

 

 

 

 

PRECAUTIONS FOR THE USAGE OF VACUUMS ON ELECTRONIC

EQUIPMENT SURFACES

 

WARNING – Certain precautions apply whenever one uses devices obtained from other than a telephone switch equipment manufacturer. This is particularly true of motor driven appliances.

 

1. Care must be exercised so that the ESD grounding points of the device, operator, and power source are compatible with the units being cleaned.
2. The operator, equipment being cleaned, and the device the operator is using on the equipment should all be connected to the same ground point. That is, the operator’s grounding wrist strap and a grounding strap placed on the vacuum cleaner brush should be as close to the tip as possible if no connection is provided on the brush itself. This configuration with standard brushes still does not allow vacuuming of live components. The ESD dissipating brushes should be used during the cleaning of internal surfaces on a powered frame.
3. Inadvertent bumping of the grounded, metallic parts of the motor housing into equipment at different ground points must be avoided. Insulating, plastic housings are preferred.
4. Many devices, specifically vacuum cleaners, have fans that will blow air out of the exhaust. This exhaust should not hit uncleaned surfaces where inadvertent secondary dirt sources could be generated.
5. The proper procedure for cleaning is to start at one corner or side of a particular electronic equipment area and work away from that point so that the secondary air source coming from the motor only hits previously cleaned surfaces.
6. Vacuuming of powered electronic equipment should NOT be attempted if the relative humidity of the space is below 25% RH.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Section 12.0

Conclusion

 

 

 

 

 

 

 

 

 

 

 

 

This report details Telcordia Technologies recommendations for equipment restoration and replacement located on the various floors of 140 West Street.  The recommendations presented are based on comprehensive physical and chemical analyses and testing with reference to Telcordia Generic Requirements, manufacturer data, and vendor and industry standards for the reliable operation of network equipment. Telcordia has also presented concerns that may warrant the replacement of select equipment based on the inability to provide continuous network services. These include logistical issues related to equipment cleaning, relocation or inaccessibility during facility repair, interoperability and compatibility requirements, maintenance, regulatory obligations and contractual uninterrupted service assurance. Telcordia also proposes that serious, although not fully quantifiable, concerns exist with the cumulative effects on long-term reliable operation of impacted equipment. These include thermal, vibration, and environmental excursions coupled with contamination with water and particulate, corrosion and physical damages and their potential to void equipment manufacturer warranties.

 

Telcordia has also presented numerous data that shows the widespread distribution and related concerns with contamination of network and support equipment and facility with asbestos containing particulate.  Telcordia cannot emphasize enough the need for Verizon to continue their asbestos monitoring, containment and abatement efforts.

 

The authors would like to express our appreciation to all of the Veizon employees and contractors and the many Telcordia subject matter experts that have provided expertise, direction, and support for the production of this report.

 

Let us also express that Telcordia Technologies is proud to be part of the Verizon Recovery Team and is dedicated to the continued support of Verizon’s efforts to restore the compromised 140 West Street office.

 

For clarification or additional detail regarding any of Telcordia Technologies recommendations or conclusions made in this report, or for any other questions or assistance, please contact Tom Gmitter at 732-758-3396 or via his cell phone at 908-415-9739.

 

 

Sincerely,

 

 

Chao-Ming Liu, Ph.D.                                              Tom Gmitter                                     

Director                                                                       Program Manager                               

Disaster Prevention and Recovery                              Disaster Prevention and Recovery

 

 

 

                                   

Randy H. Schubert                                                   Steven J. Castellano

Member of Technical                                                  Member of Technical Staff

Disaster Prevention and Recovery                              Disaster Prevention and Recovery

 

 

                                                           

 

 

 

 

Anna Halajko

Analyst

Disaster Prevention and Recovery                             

 

Copy to:

 

Verizon:

Larry Babbio

John Barrett

Mike Daigle

Charles Dunsey

Levi Nigg

Rudy Verdes

Hugh McQuillan

Charles Romano

David Rosenzweig

Charles Yaunches

 

Additional distribution at the discretion of Verizon

 

Telcordia:

David Burns

Dennis Jennings

Françoise Sandroff

Mary Hines

William Shuttleworth

Willard Wharton

Chao-Ming Liu

Department File

 

 

Workplace bullying Week

Friends,

 

October is cancer awareness month and please let’s pray for all with this horrible disease.   This week though is bullying awareness week, as well.

 

Please view my friend Dr. Victoria Johnson’s great work from the Moral Courage Channel, as she speaks about Workplace bullying.

 

http://www.youtube.com/watch?v=w349PXuyZLA

 

Neal Dias

Journalist needs help on major Verizon Story:

 

 

A Journalist is researching the “Bob” outsourcing story that broke in January 2013 on the Verizon Security Blog. See original post here:

 

 http://www.verizonenterprise.com/security/blog/?postid=1626  

 

Anyone with information on this story (e.g. what is “Bob’s” real name?, what company did he work?, where is he now?, etc.) should contact Rene Chun: rene.chun@gmail.com

 

Mr. Chun is a New York-based journalist. He has written for The New York Times, the Atlantic, Esquire, GQ and many other popular publications.

 

I already have gained information on this for Rene, but were looking at completing the puzzle.  This is a very important and damaging situation for Verizon, so please help.  You can contact me: Neal Dias on facebook, email me at justicefor7@yahoo.com, or call me at 774-991-3663.

 

We promise to never disclose were we got this information.

 

Bob (not real name) if you can, you as well can contact me or Rene, we will never tell that you contacted us either.

 

Ty

 

Neal

Verizon Security Blog

Verizon Employees – I Need You’re Help!

 

 

            There is not a day that goes by that I am not contacted by an employee, attorneys, or the media in relation to Verizon.  Many things are coming down the pike and I’m excited to continue to expose them for what they really are and how they treat their employees.

 

I was recently contacted by a reported that was working on a story about Verizon which is very damaging to Verizon.  Once he was working on it, it disappeared and his leads dwindled away.  I was able to locate this link below and I need you to read it, and help me to help him get this story.

 

So my contact (this journalist) is researching a magazine story on this “Verizon Bob” in this article and would appreciate “ANY” information from former Verizon employees who know anything it this article below.

 

This was an article that was posted on the “Verizon Security Blog” post: 

 

http://www.verizonenterprise.com/security/blog/?postid=1626

 

If you have any information you can contact me or my contact journalist directly. 

 

Neal Dias: fightingfor7years@hotmail.com

Rene Chun: Rene.chun@gmail.com

 

 

Thank you,

 

Neal Dias

Verizon Book

Verizon Book:

 

October 1, 2013 was my due date to turn in a complete and finished copy of my book about Verizon and Workplace Bullying.  My task is complete, and the book is finished, but there is an issue.

Two weeks ago I was reading over my book, and I began to see so much hatred towards my former bosses Paul McCarthy and Patty Regan and others at Verizon including Verizon itself, for what they did to my family and I.  I did not like what I saw in my own book at times. 

The pain my family and I have endured has been beyond description, and as much as I have thought about retaliation in a major form (from time-to-time) towards these individuals.  But I am thankful for the person that I have become over the years to not have not acted out on my thoughts.  I know what I credit that too is my family.  Whenever I have any thoughts of anger that crosses my mind, I look at them and I don’t ever want to lose what I have, and have my freedom taken from me.

So I saw so much hatred, and the way it was expressed in my book made me upset at the person they made me become.

So, what I did was contact my publisher, and postpone the deadline for a few more months.  I want to remove myself from this book, take time to spend with my family, revise and improve my story then send out positive message from the hell that I endured at Verizon.

My original goal when I decided to write this book, was to tell my story of what Verizon put us through as a family, the failure in the judicial system in every way when it comes to issues like this, the emotional and financial hardship we went through and how we as a country and corporations can help to change this problem.  I eventually want High Schools, Colleges / Universities, Ethics Educators and Corporations to learn from Verizon.  I want them to institute my book as part of an educations format as well.  And eventually I will get this to a movie producer to further enhance this story.  So in order for my message to be influential I have to do this right.

I thank all of you for sharing so many of your stories with me, and like I promised many will be in the book.  I wish I could place them all in there buy with over 1,000 stories and many thousands of comment, it’s impossible, but they all will be on the Verizon Bulling site that I am having created for 2014. 

I am glad to state that I just received notice that my second book will be in the stores in two weeks. 

Please expect this Verizon book in the stores soon, and thank you as always for all your support and prayers!

 

“The only thing necessary for evil to triumph is for good men to do nothing.”
-Edmund Burke

Neal W. Dias
Fightingfor7years@hotmail.com
Justicefor7@yahoo.com

My Blog Page:
Blog Address (GOOGLE): Bullying at Verizon – Enough is Enough

Facebook:
• Justice at Verizon
• Enough is Enough – Stop Workplace Bullying

My Story:

http://justiceatverizon.wordpress.com/2012/05/05/bullying-at-verizon/