Hi, my company ships a metadata-driven 3rd-party application which creates and manages universes and WebI reports. We're finally starting to get customer interest in upgrading to BOBJ BI 4.0, so our next release will support that. Like a lot of other people, we were dismayed to learn that SAP has gutted the Java Report SDK, which was the only SDK capable of creating WebI reports. So right now I'm faced with the prospect of a release which will only create WebI reports for 3.1 customers, and leave 4.0 customers out of luck.
Here are some of the workarounds I've been considering. They are all pretty expensive/risky, so I don't want to commit to any of them until I have better insight into where SAP is taking things. I'd appreciate any feedback on any of these ideas.
1. Ditch WebI reports and build Crystal Reports instead.
i. There's still an SDK available for Crystal Reports
i. We believe our customers still use WebI more than Crystal Reports.
ii. We've not yet worked with this SDK so we can't estimate how hard it will be to use or how buggy it might be.
iii. We don't know if SAP will de-support that SDK in a later release as well.
iv. We don't know if SAP is pushing customers toward Crystal Reports over WebI, or plans to continue with both over the long term.
2. Hope for an Report SDK replacement in a future release.
i. Less to do now for us.
ii. The replacement, when it comes, may be more reliable and cheaper to use that the current Java SDK. For example, it may be supported directly in .NET so we can strip out our Java components, or it may be a true XML SDK so we can simply build reports as XML docs and push them reliably to WebI
i. Our 4.0 customers won't get any reports.
ii. The replacement may never come.
3. Hack it.
i. Use Fiddler, java reflection, etc, to emulate WebI and bypass the API.
ii. Look at BOBJ's report archive/migration utilities, or at the WebI repository, to find a way to fake it at a binary level
i. Seamless to our customers
i. Probably way too expensive and risky to really take seriously
Thanks in advance for any feedback or advice. We are of course pursuing the same questions in parallel with our SAP partner contacts.
Right -- simply coming up with a hack that didn't also void SAP support for all the other reports on the system would be a long shot at best. I wanted to put that out there though, in case other users have found creative workarounds. For example generating into a separately maintained 3.1 system migrating reports manually between systems.
We will follow up with our internal contacts and maybe learn more. My hunch is that SAP ditched this SDK because it was impossible to map its many state-dependent methods to anything meaningful in BI 4.0, and that SAP is planning to replace it with something else, probably by building out the .NET Reporting SDK functionality in sync with new Java functionality. But I'm still having to read between the lines here.
Re: Point 2 about a Replacement SDK
The Web Intelligence Product Team is planning to deliver a Web Services API in an upcoming release (after Feature Pack 3) that is intended to resolve the SDK migration blockers for the vast majority of WebI SDK customers (note: exact feature details still being worked out, and future plans are subject to change, as always).
As a Web Services API, the plan is for the API to be accessible from nearly any development language or platform, including Java and .NET, as well as mobile application languages like Objective-C for IPhone and IPad.
This API is planning to cover report consumption scenarios, including data source and prompt management, report scheduling, and viewer integration. The team is also planning to deliver additional customization options for the Query Panel User interface.
WebI report creation APIs are out of scope for the initial delivery, and are planned for a later release.
I see from your message that you're doing report creation scenarios, which right now we've put out of scope for the initial release because we believe there is less demand for creating a WebI report from scratch programmatically.
Could you describe why you are creating WebI documents completely programmatically? Without knowing about your product directly, I would think you would prefer to have a set of reports created that can be easily deployed programmatically at a customer's site, rather than re-creating the report each time.
Solution Manager, BI SDKs
Hi Terry, and thanks for that update.
I work for Noetix Corporation. Weu2019ve been a Business Objects / SAP partner for almost 5 years with our Noetix Generator for Business Objects product.
Our core products (NoetixViews and Noetix Analytics) create database views and data warehouse tables used for real time reporting and analytics of Oracle EBS systems. We also have a Noetix Generator product family, providing integration with major BI platforms (BOBJ, Cognos, OBI EE, etc.).
Noetix Generators do two main tasks:
u2022 Generate the semantic model for the BI Platform (in this case, BOBJ Universes) using metadata describing the views and tables from NoetixViews and Noetix Analytics.
u2022 Generate sample reports (in this case, WebI reports) using metadata from NoetixViews that provide report definitions in a BI-platform-independent form
Noetix Generators are admin tools and are not deployed outside of IT. The generation process is essentially a one-time step (no u201Cre-creating the report each timeu201D, just occasionally during upgrade processes) . So you are correct to some extent when you suggest that weu2019d want to create a u201C[set of reports] that can be easily deployed programmaticallyu201D. The catches are that:
1. We use the same metadata to create these reports on all BI platforms. Maintaining separate metadata for each BI platform is not an option.
2. The reports we create are sensitive to the customizations in customer EBS configurations. Itu2019s unlikely (I think) that SAP could create a tool that could export a set of reports from one custom environment to another without allowing for some significant (metadata-driven) tweaks in between.
All that said, hereu2019s what I think a report creation API should look like (and what the other major vendors' report creation APIs all actually do look like):
1. Define each report as a single XML document. Make that XML easily viewable, at least to super-users or developers.
2. Define an API or create SDK tools for exporting/importing/securing these XML documents. Good language support (.NET, Java, etc.) is nice here, but really secondary since 95% of the real work is in manipulating the XML. We can always just shell out to a command line utility for the other stuff if we have to.
Thatu2019s pretty much it. In the case of BOBJ, the XML would likely contain numerical ID pointers (rather than string references) to underlying semantic objects like Universe IDs and the columns defined in the Universe. This might be an inconvenience, but not a significant one from our point of view since weu2019re already used to working with these IDs (from our experience with BOBJ XI).
I appreciate the details in your email. Very helpful.
We actually did several Proof-of-concepts for representing a Crystal Report as an XML document (The Crystal Report Application Server SDK actually serializes its report modification calls into an XML stream), but if a user hand-edited the XML we found it was too easy to corrupt the report.
For your Point #2 ("Define an API or create SDK tools for exporting/importing/securing these XML documents"), you're basically describing our entire Platform SDK.
Are you familiar with our Platform SDK? Reports can be packaged into BIAR file and deployed to any system that uses the BI Platform. You can search the forums for BIAR, or get an intro (from the 3.1 documentation) at http://www.sdn.sap.com/irj/boc/index?rid=/library/uuid/d01ce8a3-84b8-2b10-8f9e-a5b1a35c8c54
I guess it depends how customized these sample reports are, but I've worked with a lot of partners and I've usually found that customers in a particular industry want the same 10 reports, and the underlying database structure differences can abstracted away with a carefully constructed universe/metadata model.
When a report is deployed at a customer site, you call the programmatic equivalent of "Set Datasource Location" to link the report to the datasource.
I assume by Platform SDK you mean (for .NET at least) BusinessObjects.Enterprise.Desktop..dll, CrystalDecisions.Enterprise..dll, etc. We use these in .NET for universe deployment, role creation, and so on.
Unless I'm missing something here, a BIAR file based solution would involve:
1. Create static BIAR files internally by generating all of our supported Noetix Answers (we have approximately 1000 of these, spanning dozens of EBS applications and customer industry groups) into a pre-4.0 system.
2. Build internal tools to maintain and upgrade these files to BI 4.0 and future BOBJ releases, and to periodically sync these with our Answer metadata.
3. Extend our deployment system to distribute BIAR file sets on a per-customer basis subject to their licensing of our content.
4. (nice to have) Upgrade our generator to automatically deploy these BIAR files on customer systems.
Each of these steps would represent significant development effort and cost.
But if I'm not mistaken, the BIAR files are binary and there is no SDK to modify them directly. Assuming this is the case, the solution, no matter how expensive, would not work, for the following reasons:
1. If reports within BIAR files reference columns within universes by name, they will break at many customer sites because we support arbitrary custom renaming of all of our content, via hook scripts and/or newer GUI-based customization tools.
2. If reports within BIAR files reference columns within universes by ID, they will break at every customer site because we have no control over how the Designer SDK assigns column IDs and instead can only enforce ID lineage from the original universe creation point forward.
Please let me know if there's something I'm getting wrong here. We're certainly open to new options, but currently I don't see any viable way of creating anything beyond Universes and roles for BOBJ BI 4.0.
Also, regarding XML and the need to keep people from shooting each other in the foot, you should recognize that your competition has decided long ago that that's really a non-issue. Cognos and Microsoft BI use XML for both reports and for the semantic (aka Universe) layer. Oracle OBI uses XML at the report layer and has (as of 11g) added support for utilities to export/import their semantic layer (a binary .rpd file format based on the legacy Siebel BI tool) as XML. Casual users don't have access and wouldn't dare touch it if they did, and power users are smart enough to know that nobody is going to support their manual tweaks.
Edited by: Eric Hirst on Jan 13, 2012 1:34 AM
It's a little bit of a workaround, but the problem you raise of migrating biar files can be resolved by generating the biar files through the XI 3.1 BIAR engine command line tools or by creating it through the Import Wizard UI. Then use the BI 4.0 Upgrade manager (either command line or UI) to migrate the BIAR file into BI 4.0. Finally, use the LCM tool to export them back into LCMBIAR files. There is a command line option in the LCM tool to import and export LCMBIAR files.
For the second question on how WebI reports link columns to Universe objects, I'm not completely clear on why you need to do this. If you promote a 4.0 LCMBIAR file to your customers using the LCM (LifeCycle Management) tool, the LCM tool will automatically ensure the WebI reports are linked with the correct universe after promotion.
LCM guide: http://help.sap.com/businessobject/product_guides/boexir4/en/xi4_lcm_user_en.pdf (Pg 49 Command Line Option)
Thanks to Michael Sung in Development for the background.
Update to this thread in case people reach it by searching on SDK roadmap:
The WebI REST SDK in 4.0 SP6 and 4.1 now supports getting and updating the structure of a WebI report. Details and examples can be found in the documentation at http://help.sap.com/businessobject/product_guides/sbo41/en/sbo41_webi_restful_ws_en.pdf
Look in section 3.4.7 - Report structure: getting and updating the structure (specifications) of a report.
4.1 documentation can be found at http://help.sap.com/bobip41/
Solution Manager, Business Intelligence SDKs
Thanks Terry, that looks great. Yes, section 3.4.7 of the doc describes exactly what we've been looking for. A couple more things that I don't see here that would be useful:
1. Something in WebI or another tool that allows us to quickly look at the spec for a given report without using the SDK. OBI and (I think) Cognos have similar functionality in "advanced" tabs. Not a huge concern of course since we can always make our own tool to do the same thing.
2. XSD to validate documents like the one on p.198. Again, we can easily fake this, and will likely create our own simplified XSD to drive our XML serialization classes here, but it's useful in our product lifecycle to have access to report XSD for successive BI tool (e.g. WebI) versions. Being able to compare the WebI 4.1 XSD with the WebI 4.x XSD using a diff tool will allow us to see at a glance what has changed in the spec that we might care about. (I've been doing the same thing with your Designer COM automation SDK for years now, by exporting the COM IDL to a text file that I can compare across versions to find new function signatures.)
I know the thread is answered but just to comment on the "binary BIAR" assumption, LCMBIARs (not sure about 3.1 BIARs) are just zip files with a different extension, like the new Microsoft Office formats. Rename it to a .zip extension and you'll see the different files inside.