SAP Lumira: Max Data holding Capability

[Akshay_Bhandari]

Hello Team,

We are using SAP LUmira 1.19 . We are creating reports over Bex Queries.

We need to understand the maximum data holding capalibilty of Lumira. Also in a report in case of  Tabular structure what is the maximum number of rows and coloums a report have. Keeping all the performnce Optimitaion into consideration.

Best Practices in case of SAP Lumira over Bex queries is highly appreciable.

Many Thanks

Regards

Akshay

[Karol-K · Accepted solution]

Hello Akshay,

here the answer, w/o asking why you need 12 millions cells in a Lumira Desktop application...

BW Online connectivity in Lumira Desktop is using BICS as access layer. From the logical data perspective you can pick up a result set which is not restricted at all, BUT there are some technical points which will restrict your work...

1. first one is the safety belt parameter (which you for sure hit), this is applying to all clients communicating through BICS - 
2. the available memory on ABAP (can be scaled up)
3. network, the result set needs to be transferred from ABAP to Desktop, this means your 12 millions cells must be shipped through network
4. the client memory allocated by Lumira Desktop (you can probably manage on good PC with 64bit installation, calculation can be done via note: 1177020 SAP BusinessObjects Design Studio - Sizing Information (applies to Design Studio, but also to Lumira BW Online as BICS is in place)
5. finally the visualization - I have no clue how you want to visualize 12 millions cells (800.000 rows). Chart cannot make this, Table will start to make "paging" internally, so you will be never able to see the complete table...

Now, I would like to ask the question - why?

Karol

P.S. of course if you have additional questions, just ask.

[Akshay_Bhandari · Accepted solution]

Hello Karol,

Thanks for such a descriptive explanation.

We were making a feasibility report of the product which will meet our client requirement.

We wanted to understand what is the maximum number of rows*columns in a tabular structure can Lumira Or  WEBI hold so that we can meet the requirement and provide a clear picture to the client as N*M is the maximum number of rows*columns the product can display.

If case you have any information on the Maximum data holding capability of WEBI or any settings that would help us to achieve the same will be of great help.

Many Thanks

Regards

Akshay

[Former Member · Accepted solution]

Hi, Karol.

great stuff as always. 

Since you are the design studio guru as well, need a confirmation on the safety belt:

Does it apply to design studio and lumira when connected to Bex queries?

According to the famous how-to paper, safety belt only applies to webi, not analysis for OLAP.  Hence the question.

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/20681f42-5be5-2f10-f890-dc8578e6e...

page 10.

Thanks.

[Karol-K · Accepted solution]

Hi Peter,

I have checked the settings in Design Studio and Lumira BW Online case. I have decreased the safety belt to 20 cells and none of those those tools have applied. On the same time, BEx portal connected to the ABAP stack has applied the change after RSADMINS were reseted in Java.

My understanding was that all tools which are connected with BICS are evaluating the safety belt on ABAP stack and the supporting topic was only by overwriting the backend setting, but this seems to be incorrect. I have to double check this and ask some colleagues, as at least in Lumira BW Online I would expect the safety belt applies to the result set.

I will get back to you after some more tests in ext days (next week probably).

Karol

[Former Member · Accepted solution]

Thanks for taking the time, Karol.

Your result is somewhat expected, as it confirmed the same in the attached SCN how-to guide:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/20681f42-5be5-2f10-f890-dc8578e6e...

page 10.

The Analysis OLAP safety belt is apparently set on the MDAS APS on the BI platform.  When I wrote to you, I suspect that is where design studio and Lumira is getting the safety belt as well.  Unfortunately I don;t have a CMC in front of me to try right now.

Thanks again.  Looking forward to your findings.

Peter

[Karol-K · Accepted solution]

Hi Peter,

I made some research. My information was incorrect. The safety belt is implemented in BICS data access layer, but the clients must read it out from corresponding ABAP and pass it to BICS. And as of today Design Studio and Lumira BW Online are NOT doing it. This means, in those clients there are no restrictions on size - in worst case something will break with OutOfMemory.

Shortly, Lumira BW Online and Design Studio does not support safety belt settings.

Regards, Karol

[Henry_Banks · Accepted solution]

Thanks for taking the time to check that!

[nmsr1976 · Accepted solution]

Hello Karol!

how is Design studio and Lumira today, regarding to safety belt settings?

Thks for your time.

[Former Member]

We are on 1.29/BIP 4.1 SP5 and noticing odd behavior that we think is tied to dataset size.  Fairly certain we are not in the realm of anything extreme, but will see if I can get info. 

Essentially, using SAP BW (BICS) we will see that at an undetermined threshold Lumira does not even seem to kick off any background processing on BW.  Lumira will just appear to process for some time on the desktop, then give us a blank visualization with no data in  'Prepare.'  No error, no message, no apparent activity in BW.

We remove some of the dimensions, restrict some data and then it works great.  Running additional test scenarios...just throwing out there in case anyone notices the same.

It is almost like Lumira will not even send the request to BW when certain conditions are met.

[Former Member]

Hi,

In Lumira Desktop we can specify the max limit of rows and MaxPermSize. In default

MaxPermSize=128M

and maxvizdatasetsize=10000.

Even we can extend these values.

Major problem is .ini file gets stored in C:

[Former Member]

hi

For Lumira Cloud, we limit dataset uploads to 200MB and incidentally 1000 columns.  I’m not aware of a limitation in # of rows. The desktop version only uses your local computing power and doesn’t interact with HANA. Uploading a huge dataset to Lumira Cloud would take a while due to network & other factors, but once uploaded you have HANA Cloud efficiently doing queries, whereas the desktop version will be able to get at the dataset quickly but isn’t a highly optimized database for doing the big data queries.

I hope this helps.

[Henry_Banks]

Wow a 1000 columns! that is very very wide!  FYI - we've been testing performance for 300 cols use-case, and assumed that was an extreme case 😕

[JayThvV]

By default, as defined in Lumira.ini, Lumira displays a maximum of 10,000 data points per chart. You can manually change this number if you really want, but that gives you an indication of what is a reasonable amount of data to visualize.

[Akshay_Bhandari]

Hello Jay

Thanks for the important information.

It would be of great help if you provide some information on the number of Rows*columns that can be displayed in the tabular structure..

Many Thanks

Regards

Akshay

[JayThvV]

I am not sure there explicitly is one. There isn't anything like it in the .ini files, although it does allow for increasing the VM size. The same is true for WebI, as well. There aren't any hard limits to my knowledge, at some point everything is just going to run slower and slower, and take more and more memory.

I'd look at it from a best practices perspective: in both WebI and Lumira, you would want to have the database do as much of the work as possible, and pre-aggregate as much as you can before it is inside the BI or visualization tool.

So, rather than looking for a single tool that can handle larger datasets than others and push the limits, think about the use cases where you would use any particular tool. Considering - with modification - you can only show up to 10,000 data points, why would you want to upload millions of rows of data? If you want to navigate interactively along a cube with a number of different dimensions, especially if in a hierarchy, Explorer is likely a better option, for instance.

[Former Member]

Jay

Good feedback.. Im faced with a similar use case, the users don't trust the data, dont' have specific questions, and want 25m records to be dumped into access from hana to perform the validation, and examine various questions based on their understanding of the data.

My assessment:

1. These users have a data miner profile
2. Are we "selling" the wrong product to our user profile (data miner/analyst)
3. What other techniques could we deploy to address their concerns (data trust, open ended, boundless questions)

Thanks

johnK

[JayThvV]

I would argue, the wrong tool is often the case. I have ran into a number of accounts, for instance, where there was a sentiment of "standardize on WebI" for all use cases. The reason was so people didn't need to learn more than one tool...

Different user profiles require (often) different tools. If you're a data miner, something like Explorer makes more sense, unless you're very handy with SQL and then perhaps we may even give you HANA Studio - after some clear instructions and understanding of the impact your SQL may have on the overall system.

Often the "just give me access, and I'll do it myself" shows up a clear problem between what the business needs and what they're getting from IT. Better communication and perhaps some governance structures make sense in that situation. Often, I think, a full self-service approach is a bit like throwing up your hands in defeat and let the users figure it out by themselves. But it often leads to 1,000s of users essentially running and developing the same reports. There is something to be said for a collection of pre-built, validated reports - especially for those key metrics that operations are being run of.

If users don't trust the data, you have a much bigger problem.

[Former Member]

Jay

Thanks for the input..  The business wants more speed, and autonomy.. And we've done that with Qlikview via self service, but finding the build out in BW/HANA to be painful.

The idea of guided reports & analytics that are validated / certified could be the key..

John

[achab]

Hey Akshay,

I am not sure to get your mention of "maximum data holding capalibilty of Lumira". In the current integration on top of BW Bex Queries, the data is not moved from the BW system to Lumira. It remains only in the BW system.

Best regards,

Antoine

[Akshay_Bhandari]

Hello Antoine,

I wanted to know in case of a tabular structure what is the maximum number of rows and coloumns a report can have.

Many Thanks

Regards

Akshay

[achab]

It's a good question but neither the community (I searched) or I have relevant information on this aspect. I'll pass on to the experts. Can you give us a feel of the typical data volumes (#rows, #columns) you would like to handle?

Thanks

Antoine

[Akshay_Bhandari]

Hello Antoine,

I too had searched for this in the community and in the User guide even i didnt find any information.

We we looking out for aroung 8 lakh records i.e #Coloumns : 15  and #rows : 8 lakh.

We had a similar requirement for SAP BO WEBI. any information on this also will of great help for me .

Many Thanks

Regards

Akshay

[achab]

Hey

So 800.000 rows * 15 columns = 12 millions cells, right?

Thanks

Antoine

A lakh or lac (/ˈlæk/ or /ˈlɑːk/; abbreviated L) is a unit in the South Asian numbering system equal to one hundred thousand (100,000; Scientific notation: 105). In the Indian numbering system, it is written as 1,00,000

[Akshay_Bhandari]

Hey Antoine

You are right

1 lakh = 1,00,000

800.000 rows * 15 columns = 12 millions cells

Many Thanks

Regards

Akshay