Salesforce Outage: When On-Demand Isn't

Gwynne Young
Managing Editor, CustomerThink
Member

Posted 21-Dec-2005 09:05 AM
ComputerWorld reported that salesforce.com had an outage Dec. 20, with many users unable to access the program for much of the day.

This would have to be a huge worry for on-demand providers and users. Because it made the news, I imagine it's not something that happens everyday.

Does your company consider reliability when it's considering an on-demand solution? What about internal web access? Your provider can be exceedingly reliable, but if your own company's network has problems, it might not be worth it.

If you're a CRM consultant, do you bring up the issue of reliabile connectivity when you're helping your clients weigh options?

Graham Hill
Guru
Member

Posted 21-Dec-2005 11:21 PM
Gwynne

Reliability in terms of system availability has to be an issue when planning any systems implementation.

But it shouldn't be any more of a problem if you are using an on-demand product delivered over a public or external-private network than if you are using an installed product over an internal-private network.

The key questions are always around availability (how do you gaurantee that the application is always available), loading (how do you gaurantee that the application is always responsive no matter how many people are using it), overflow (what happens when too many people are using it) and back-up (how do you guarantee that prior and current work is protected in event of a catastrophic failure).

You should ask the very same questions no matter who is providing/operating the application and whose network is being used.

You may find that your own IT department is often less able to answer these questions and provide the iron-clad guarantees required (with sanctions for failure) than external providers.

Graham Hill
Independent CRM Consultant

Pierre Hulsebus
Member
Picture of Pierre Hulsebus

Posted 05-Jan-2006 08:26 AM
During a recient engagement with a major Micigan pharm. I was confronted with this issue. Our calculations indicated that when the CRM application was unavailable to the sales person, the cost was $5,000 (US) per day. That was the sales impact when a sales person was without a laptop. When the CRM app crashed the remote sales person had to send the laptop back to support for reindexing and rebuilding.

It steered us to require that the laptops be configured with SQL and not a flat file structure offered by a particular CRM app, GoldMine in this case. This required additional ram on each machine, change in the the license and higher IT costs. However we were able to see a substantial increase in up time. And the recommendation not only saved money, it improved user confidence in the application.

If you said your application was 99% available you are also be saying to your users you can count on the system being unavailable 1% of 365 = 3.65 days a year. Is that acceptable, and what are the costs? What happens when those 3 days are on a month end? or Year End? This type of down time would affect order taking, sales figures, actual billed revenue.

We have to remember this is software too.... We have to get this out and on the table.

---------------------
Pierre Hulsebus
CRM Consultant
www.ehtc.com
Infuse Rocks!

Bob Thompson
Founder, CRMGuru.com
Member

Posted 06-Jan-2006 11:01 AM
No service is 100% reliable, but increasingly we expect that, whether it's DSL, cell phones, or power.

Moving the service in-house doesn't guarantee 100% uptime, either, as Graham and Pierre pointed out. The question should be whether an internal or external provider can do the best overall job reliably delivering the function and performance needed at the right cost.

That said, I do think that having a lot of subscribers concentrated in one vendor like Salesforce.com, with a hyper-growth business model, poses a risk that customers should consider.

Microsoft has been knocked for having a single-tenant architecture with Microsoft CRM 3.0. But perhaps one side benefit of the one customer/one server approach is distributing the points of possible failure.

No matter how well planned, there's a learning curve in building large scale IT systems. Make sure you're confortable your on-demand vendor has the right plans and experience. Even if your business is small, can the vendor handle other large customers that could come on-stream at any time? Or workload requests that can be triggered by an end user running a report that requires complex SQL queries and slows down everyone else.

Bob Thompson
Founder, CRMGuru.com

Jeremy Cox
Member

Posted 12-Jan-2006 07:49 AM
I share your fears Bob, particularly if a firm is moving to a deeply integrated approach to CRM. The idea of outsourcing something so vital to the health of a business fills me with dread.
A sudden ouotage will affect thousands of firms simultaneously—what chance then of getting your voice heard?

I think prudent firms who are going the integrated route in order to provide a consistent and high standard of customer service and experience and gain and use insights into customer behaviours, will want to ensure that they have a good back up policy in place. There are plenty of good disaster recovery specialists around—and I'd be inclined to seek their advice rather than risk all on 3rd party. Of course if its nothing more than contact lists for direct marketing, then it won't matter so much.

Jeremy Cox MA DipM
Managing Director
The Wisdom Network Ltd,
www.thewisdomnetwork.com

Wendy Collett
Member

Posted 12-Jan-2006 12:09 PM

Confusing situation. A reindex and rebuild of a GoldMine dBase version should have taken only 5-20 mins, and could have easily been performed by a user, or by remote access via the IT dept. You are saying that the computer had to be 'sent back to support'. Perhaps your statement was an over-simplification of the situation.

GoldMine is not a flat file database, regardless of whether it is hosted in dBase of SQL, it is a relational database.

The SQL version of GoldMine is more stable, no arguments, although it can have issues, same as any program.

However, the real point here is that there is no such thing as 100% reliable, and we do ourselves and our clients/users no favours by trying to promote this.

We would do far better as consultants to promote, train and advise, jump up and down, do whatever to do simple things like:

* BACKUPS and other disaster recovery methods. EXPECT that BAD THINGS will happen, and plan accordingly.

* Train users to deal with simple issues, and give them expectations (I have to tune my own TV/video/radio etc, Train users to EXPECT to do some simple things themselves where possible. I know, not always possible, but 'users aren't always losers' unless you expect them to be so. However, more importantly, it's hard for a lot of CRM people and IT people to actually relate and simplify computer issues and workings to users, and this is an area which does need to be worked on.

* Lastly, expect that in cases, no matter what we say or do, there will always be those that expect impossible things like 100% service. Try your best with those, but be prepared for the inevitable problems, especially those totally out of your control—eg natural disasters etc.

Wendy

Pierre Hulsebus
Member
Picture of Pierre Hulsebus

Posted 30-Jan-2006 07:04 AM
Not to cut the point too fine on GoldMine but for clarification.

GoldMine laptop uses Borland Database engine. The entire database consists of a family of .dbf files. If these files remain open during a hardware failure, or other system glitch, they are subject to massive corruption. Running a simple reindex fixes these problems very often. However, if there is corruption in the table, reindex and repair may not fix it. This will require the assistance of a professional that has to review the databases to manually find the corrupt records via a toolset like GoldBox.
Most GoldMine consultants are familar with this process. And this cusotmer was familar with it too.

The particular client's IT department was not able to resolve these applicaiton specific issues alone. As the IT supported core apps and network connectivity. App specific support for the CRM app was tier 3 support handled by GoldMine Consultants who did not have access to the internal support desk remote support apps.

So as you stated my point is a bit of an over simplification of some complex GoldMine specific issues.

The point here as you stated was to work with the client to weigh the cost of upgrading to SQL on each laptop out in the field with what was the longterm benefit of this to an international sales team who could not stand to be without access to the CRM data.

SQL was recomended, and over the next year or so as the machines cycled thru the IT department for normal maintance and upgrades they were upgraded to SQL.

We saw measurable improvements in system errors related to Database errors.

Happy Clinet..

---------------------
Pierre Hulsebus
CRM Consultant
www.ehtc.com
Infuse Rocks!

CRM Coach
Member
Picture of CRM Coach

Posted 30-Jan-2006 07:28 PM

I agree with Graham on this...I know that when there's a lot of sales hype, it may be easier for one to lose sight of the reality of *any* system.

A few days ago, I did a blog entry on the Salesforce.com outage (http://thecrmcoach.com/blog/index.php/2006/01/26/recent-salesforcecom-outage-should-be-taken-with-a-grain-of-salt/) and offered a few suggestions:
If you're going to go with a hosted provider such as Salesforce.com, here are some things to consider (especially for on-demand services):

- Can you handle being out of control of the system? especially when it goes down?

- What is the up-time service record like for the past year?

- Are they growing faster than they can service?

- What are their policies for doing maintenance and upgrades? Do they do it off hours (to you…your midnight is someone else's morning)? do they provide ample notification?

- How do they communicate with customers when the unexpected happens? Are they forthcoming?

- Is there an off-line contingency available (such as a local synch'd copy)?

Scott Gingrich
The CRM Coach
(519) 538-4619
Author of the "Insider's CRM Success Toolkit" that holds nothing back and gives you the hands-on detail you need to succeed with CRM. Free reports with Insider CRM Street Smarts.
http://www.thecrmcoach.com