Interactive analysis probe benchmarks / Expected Report Open Times

[Former Member]

Hi, can anyone comment on expected time to open a report in BO4? A lot of my users are currently complaining about "in-app delays" at the moment - anecdotally when opening a report it can take up to 30s or more. I've observed this latency myself and have been trying to gather some metrics to get an objective view of the user experience. I would be interested to hear other users experiences to guage how to resolve the issue or explain to my users that it is to be expected. Sites I've come across like this http://www.nngroup.com/articles/response-times-3-important-limits/ would suggest that 10s is the limit - but should I be able to get it down to say 3s? (Only on a single node environment, or a distributed one?)

These are my own findings so far:

I have scheduled the Interactive Analysis probe every 5mins on my Dev and UAT environments (UAT replicates Prod). In Dev I get a steady response time of 9.5-10.5 seconds. The low values in UAT match this but there are several spikes of 20s and a couple at 40s with averages of 11 - 13s. The report that the probe opens is not refreshed or exported, is set to run on demand, has 2 prompts, is the same in both environments and is 189KB on disk.

Both environments are BO4 SP4 Patch 4 but Dev is a single node install (fileserver on the same server with the application) with a remote CMS in a different domain behind a firewall whereas UAT is a clustered installation with two virtual webservers in a DMZ, two physical application servers behind the corporate firewall sharing a filestore in the same domain as the application servers but on a different virtual machine and sharing a CMS database in a different domain behind another firewall.

As well as any general comments, I would specifically like to know if:

1. The interactive analysis probe is doing what I think it is doing - i.e. opening my report, waiting for the prompts to display, cancelling and closing the report; and that the time recorded reflects this?

2. Is the limiting factor my environment or the application - i.e. if I had unlimited funds what could I get the time down to?

Thanks,

Colin

[Toby_Johnston · Accepted solution]

Hi Colin,

There are a lot of variables that determine how long a report takes to run.  Generally speaking though your right, for view on-demand requests the user really shouldn't wait more than a minute.  In these cases you should opt to schedule the report on a recurring basis or allow the user to schedule the report and view the instance once completed.

With that said, I've seen customers where their end users wait for almost an hour to get back an on-demand Webi doc.  It really depends on how you train your users.

To determine where the time is spent during a report refresh you should use end to end tracing.  An end to end trace will allow you to see how long each component in the workflow takes to process the requests.  Use the GLF Viewer to view the end to end trace.  Refer to this blog for details. 

http://scn.sap.com/community/bi-platform/remote-supportability/blog/2012/11/06/how-to-generate-and-c...

If you have SAP Solution Manager you can use the E2E Trace Analysis tool here as well and get a much richer E2E view of the workflow.  This tool also integrates http tracing with Wily Introscope transaction trace.  Check out the video link for a demo.

The answers to your questions:

1. The interactive analysis probe is doing what I think it is doing - i.e. opening my report, waiting for the prompts to display, cancelling and closing the report; and that the time recorded reflects this?

A)  Yes, except there is no cancelling that happens.  The report runs or fails and the time it took to refresh is recorded.  Keep in mind when you view on-demand there is an additional step of HTML rendering that takes place on the appserver so a real VOD request will take a bit longer than the probe.

2. Is the limiting factor my environment or the application - i.e. if I had unlimited funds what could I get the time down to?

A)  There are many things you can do to speed up the refresh such as (optimize the SQL / MDX, tune your database, tune your BOE processing servers, tune your CMS, tune your application server, add more CPU/memory hardware where needed, use SAP HANA , etc)

You could try opening a support message for some guidance.

Regards,

Toby Johnston

SAP America, Inc.

[Former Member · Accepted solution]

Hi Toby, thanks for the quick response and the info - I will look at the GLF Viewer in detail and update with my findings! I need to clarity the issue though - my users are happy enough with report run times (at the moment), their specific gripe is the time taken to click on the document tab, double - click on a report and be presented with the prompt screen to enter their parameters. It is this workflow that can take up to 30s - after they hit run they can get results back faster that it takes to get to the prompt screen. How long would you expect it to take to open a report and be presented with a prompt screen?

Also, given that my report has prompts and there was no option to enter them I set the "Refresh" property on the probe to "false" - is the report simply failing in this case? I think the loading of the prompt screen is a significant part of the latency the users experience so it is important to know if this is reflected in the probes results. If it is not, the actual times users can wait to see a prompt screen is even greater than 40s.

Regards,

Colin

[Former Member · Accepted solution]

Hi Toby, I've gone through the E2E post you supplied - great blog ! (I've rated it as well). Using the info I've successfully traced this issue in GLF and identifed 3 major latencies in opening a report to the prompt screen - all from one of my webservers to one of my application servers (7.8s, 3.4s and 8.2s). I'll continue to investigate to see if I can glean any more information to help me bring these down but if you have any suggestions please let me know.

The overall time taken to open the report was 31s. I forgot to mention in the original post and followup that the UAT & Prod systems use HTTPS which I understand from James Rapp's posts could be a factor as caching may not be available.

Thanks,

Colin

[Toby_Johnston · Accepted solution]

Hi Colin,

What type of report are you using and what is your datasource? 

There are a number of things you can do to speed up the prompt screen. 

Are these prompts based on a list of values in a universe?  If so, you can make some optimizations to speed things up.

1.  By default LOV point to the same object as the object they are attached to. But if this object points to a large table (number of rows) then refreshing the LOV may be slow. If there is an alternative smaller or faster table that returns the same values, then the LOV should be edited to point to that alternative table.

2.  Do the values in the LOV change often?  If not, you can set them to be static and not refresh against a database.

3.  If you go to the object itself in the universe, you can turn off the option to "Automatic refresh before use".

If they are based on a Crystal Report you can also create a LOV that can be scheduled (inside the Business View Manager).  With this type of list of values the list is refreshed automatically at a specific interval (once a day for example) and the end users see a cached version of the prompt list (making the prompting almost instantaneous).

Thanks

Toby

[Toby_Johnston · Accepted solution]

If your probe does not refresh the report then it is not an accurate representation of the end user workflow.  It will most definitely return much quicker (either failing or returning an empty report template)

Thanks

Toby

[Toby_Johnston · Accepted solution]

Hi Colin,

Refer to my previous post about optimizing the list of values prompts.

In the case of SSL the key here is to reduce the amount of handshakes the client needs to make with the server because each handshake there is a delay while the secure connection is established.

You could try to reduce the amount of handshakes by increasing the array fetch size on the universe connection properties since more data will be sent in each batch.  You can also set the chunk size parameter inside the cs.cfg properties file on your nodes hosting the webi/connection server services.

Regards,

Toby

[Former Member · Accepted solution]

Hi Toby, the report is a webi report built on a multi-source universe with both sources pointing at SQL Server 2012 databases. I will experiment with the prompts but I don't expect this to resolve the issue as two of them are date prompts that present a calendar for the user to select a date from and the others force the user to retrieve the list of values from the database, after the prompt screen has displayed. Neither of the 'search db' prompts are set to "Automatic refresh before use".

I have worked with our network team today to monitor network performance while I duplicated the workflow and they saw nothing unusual, all interactions between my servers took place in <1ms which was expected. The trace however again reported large latencies of up to 7s at a number of steps so I've opened a case with SAP support to investigate further as well.

Thanks,

Colin