explanation BIS outage on facebook by official RIM BB account

[Delil]

The messaging and browsing delays being experienced by BlackBerry users in Europe, the Middle East, Africa, India, Brazil, Chile and Argentina were caused by a core switch failure within RIM’s infrastructure. Although the system is designed to failover to a back-up switch, the failover did not function as previously tested. As a result, a large backlog of data was generated, and we are now working to clear that backlog and restore normal service as quickly as possible. We apologize for any inconvenience, and we will continue to keep you informed.

[CrackberryBrandon]

This is horrible timing for this to happen with the release of the new iPhone this week.

[BBThemes]

and the irony is.......people on fb are still posting their PIN`s on the fb page haha

[JasonM.]

Outages are always unfortunate, but the question begs to be asked: Why is there one core switch responsible for routing traffic for all users in that geograph? That's a pretty widespread outage to blame one switch on... I'm not going to pretend to know RIM's NOC architecture but I'd at least imagine there is more than 1 piece of anything; switch router or otherwise for that whole area.

When are they going to learn to just make the phones smarter so outages aren't so catastrophic? Use device APN and add failover imap polling in case of a loss of connectivity to the BIS? There are lots of ways to guarantee high availability, just not when you put 70M devices out there and point them at one spot...

[kojita]

agreed with previous post, this sounds crazy. they cannot possibly rely on one switch for the emea...i d be interested to learn a bit more about rim network to better understand this massive outage.