Yesterday, I announced a delay in the release of our new platform for the SAP Community Network (SCN). A number of people have asked for more details and background on this decision, so this follow-up blog is my attempt to answer as many questions as I have heard or can anticipate.
There are a number of technical platform issues that still exist, that are either critical or very important. At the tail-end of a project as it approaches the launch date, I would expect to see the severity of issues declining from “show-stopper” to “critical must-have” to “important” to “nit-picky like-to-have”… but with just days to go until launch, we were still seeing too many critical issues or bugs, and sometimes the fix to one problem caused another. For example:
- Search Indexing – This is not working properly, which means that our search engine within the new SCN cannot index all of the content in the community. This matters because about half of our community members currently use search rather than browsing and navigating to content, and that use of search will be even more prevalent on a newly organized site where everything is in a new place. Getting a fix to this might be possible quickly, but the actual indexing of 8.4 million discussion forum posts, 2 million forum threads, about 20,000 blogs and their comments, 2.8 million member profiles, a content library of tens-of-thousands of documents, and much more takes nearly 48 hours simply for the indexing to run. Without the indexing, the search engine will not function properly, or will take excruciatingly long to find content. We ran out of time to fix the indexing problem and to run the search indexing itself.
- Load testing – We cannot run load testing without all the pieces in place. Sure, the system has performed amazingly well in test environments and earlier configurations – fast and stable – but you never know until you can put the full system under stress with all of its content, and we sure don’t want to find out about problems, or be fixing them, after go-live. Load testing and search indexing cannot happen simultaneously, so that needs more time.
- Delta content migration – We’ve moved 8 years’ worth of blogs, forum discussions, member data, points and categories, and much more into the new system already. But new content is produced by community members every minute, so there is a delta – or difference – between the old and new system. We need to move that content over just prior to go-live so the new system has every piece of content that the old one did when we make the switch. This may only take a few hours, but is a critical few we don't have.
- Blog issues – We highly customized the blog capability for the new SCN in order to reflect the unique needs and expectations of our community. Points, user profiles, categories, comment capability, share on public social media, etc. But we are seeing problems with even the blog authors’ ability to move their blogs into the correct topic categories, to edit their blogs, to format them easily, and sometimes even to access their own blogs (they show up in search, but trying to get to them gives a “not found” error). This is not acceptable since blogging is one of the most important parts of our community. Our customization needs additional work to fix, test, verify.
- Single Sign-On – Authentification and access controls have a few challenges, and we think we have a fix to single sign-on, but the fix was not verified by Friday morning and we would need to implement and test the fix, but ran out of time.
- Security – A few issues still remain, cross-site scripting is not resolved, and the spell-checker needed to be disabled. Fixes are do-able, but not between Friday and Monday with enough confidence to let the launch proceed.
- Functionality – The SCN platform is very complex and inter-connected, as you might imagine. We get many new features and much new functionality in the new SCN platform, and we added or customized some of it. There is no one show-stopper critical problem with usability or user experience or the features we have in place, but there are too many medium-importance or nit-picky issues we want resolved before we go live. Alone these are nothing more than minor annoyances or items that could be improved after go-live; taken together they are a collective issue we don’t want to expose to our members and create the misperception of sloppiness.
NOTE: I don’t believe any of the above issues are inherent in the underlying platform. Rather, they are often caused by the custom code we’ve written to provide new or special functionality, or to link interdependent system components. Compounding the challenge in fixing these issues is the fact that most of our IT development team, architect, QA team, and IT project manager are in Germany, while a vendor/partner is in San Francisco, and our global team is dispersed around the world, so the time zones and turnaround are logistically difficult. By the time issues are found and communicated, fixes are developed, handed back for implementation and testing, and then personally tested by team members, we’ve seen 24 or 48 hours pass by.
IMPORTANT: These kinds of issues are to be expected in a build / migration / launch of a system as complex and massive as ours. Our SAP IT team have been and continue to be true partners who exhibit extraordinary expertise, dedication, and commitment; we can ask for nothing more than to have these professionals at the lead. Likewise, our external vendors/partners are top-notch, and extremely engaged and supportive. We simply ran out of time relative to the complexity and scope of the tasks at hand. I have full faith in, and the utmost appreciation for these people, and I know that together we will deliver an extraordinary new SCN soon … we just need a little more time.
Members of the team noted their willingness to cancel vacations and work through the holidays in order to deliver the new SCN faster. But I know how hard these people have been working over the past year; I've seen them and followed their planning and execution discussions. I know the sacrifices they’ve made to work/life balance, their stress levels, the degree to which they’ve committed themselves, even the sacrifices their spouses and kids have made.
I am not willing to put this project above or ahead of important needs of our people. To the contrary, I believe that the team members will perform more effectively and will deliver better over the long-term if they get a much-needed break. I also want them to go into our launch full of excitement, positive energy, and enthusiasm for what they’ve accomplished rather than exhausted and drained. This should be a collective celebration when it happens. Our people are more important than a target go-live date.
Many on the team have plans for Christmas or New Year travel, breaks, family time. So do other colleagues across SAP who may be second- or third-tier contributors, but whose relatively small contribution might be a show-stopper if they are unavailable at just the right time. While we’d love to get the new SCN launched by year-end, we face too much risk due to holidays and vacation plans at this time in the year.
Overwhelming Community Member Support
One of the most heartwarming and encouraging aspects of announcing my decision to delay the launch is the outpouring of support from valued members of the SAP Community and from analysts and influencers, colleagues, SAP Mentors, and active contributors I genuinely respect and admire. Here is just a quick taste of my recent Twitter stream on this topic:
Is that not amazing? Seeing the trend of “tough call, we're disappointed, but we appreciate the emphasis on quality and support the decision and the team” messages is really extraordinary. I can’t say enough about how much that means to us all.
Of course, all reactions have not been positive or understanding. Especially sharp and biting criticism came via a very negative ZDnet blog. Part of the reason for this very blog from me is an attempt to fill-in the gaps in information in order to satisfy that type of tough critique and others that might go un-written. I hope my follow-on blog helps somewhat, although I’m not expecting that I can answer every criticism adequately enough to satisfy all of the critics despite the length and scope of this. There may be some other harsh reactions out there already or brewing, but I haven’t seen them yet. They’ll come, I’m sure, as will the positive and supportive notes and write-ups.
There’s often good that comes from any challenge or disappointment, and this experience is no exception. For example, we will use the brief delay in launching the new SCN in order to invite our SAP Mentors, some of our platinum members, moderators, and a few other key stakeholders to participate in a private, pre-release beta of the new community platform. This is something we had wanted to do but couldn’t fit into the schedule due to timing; now that we have time, we will invite a small group to test features, functionality, ease-of-use, design, overall user experience, and much more. This is a very exciting opportunity for us. The rest of the community will benefit by having true experts and the most-active SCN members pre-testing and vetting the system while our team still has time to develop and implement many of their suggestions before we release it, live, to the world. Ultimately, this short delay will allow us to build a better SCN pre-launch than we could have otherwise.
Some will ask: Who’s responsible for this delay? Who needs to be held accountable? Who can we point a finger at for this delay? Easy answer: the buck stops here, with me.
Therefore, I ask that you please continue to give your full support and appreciation to the hard-working team behind SCN. That includes the folks who work with internal SAP groups to source, publish, and curate content, those who interact with active community members, organize events, build and run the underlying system, promote the community, aid new members, moderate discussion topics, and do a wide range of other actions daily in order that the SAP Community can run, function, evolve, and best serve our members. The dedication, commitment, and expertise of this team is beyond measure, and they deserve your support as they bring us this last mile down the home stretch, and afterwards as the new SCN is in full production mode.
I’d like to reiterate what was most important in deciding to delay the launch of the new SCN:
1) The quality of the SCN platform – including content, capabilities, performance – is of utmost concern to us, and it just wasn’t good enough to meet SAP's standards or ready to be deployed in production for our 2+ million members.
2) “Do what’s right for our community members and customers” is our main guide (versus getting something out within a certain fiscal year, hitting personal goals/KPIs, or achieving somewhat arbitrary target dates ... and despite the pending embarrassment or disappointment). We’ll take the hit on criticism in the short-term in order to do what’s right for the customer community in the long-term.
What to expect ahead
We expect to go live in early 2012 when feedback from our beta testers and our own quality and performance tests indicate that the platform and the community functionality is ready. I don’t think you’ll have to wait very long but I won't venture a guess publicly since I don't want to set wrong expectations we might not meet.
Until we do go live, you’ll hear updates from me and others when there’s news to report. Some of the intangible aspects of our community that make it stand-out from others is its transparency, openness, authenticity. We will share with you frequently and openly and candidly.
Again, I apologize that we’ve had to delay our much-anticipated launch. It's disappointing to you and to us. Yet, I am even more convinced that we made the right decision, since I’ve heard kudos from many of you, and I know in my gut that putting quality above an on-time launch will serve us well in the long-run.
We will continue to work hard on making this a fantastic new platform for you and other SCN members, and we’ll work fast so you won’t have to wait too long. Thank you, again, for your understanding and support. I and the team really appreciate your trust and confidence.