cancel
Showing results for 
Search instead for 
Did you mean: 

Queries on Memory measurement and related

Former Member
0 Kudos

Hi,

Below mentioned are some queries based on the articles I have come across about HANA especially the HANA memory usage explained document.

Link-> http://www.saphana.com/docs/DOC-2299

I request your help on these queries.

Some terms:

1.  Physical Memory(RAM): The amount of memory available on the machine.

2. Swap: Disk space that can be used in case physical memory is full. 

3. Allocation Limit: Amount of memory that can be allocated (reserved) from the operating system (maximum pool size).   ·         

4. Allocated Memory: Amount of memory currently allocated from the operating system (current pool size). This is the memory available to HANA database.

5.Used Memory: Amount of memory that is actually used by HANA Database (share from the pool that is in use, this includes memory used for the program code, the tables, and for temporary computations)

6.Virtual memory is the Physical memory(RAM) +Swap space(Disk) that is allocated for a process.

Queries: (with the understanding that the terms Used memory, Resident memory, Allocated memory, HANA memory pool, a part of virtual memory(non-swap spcae part all refer to RAM)

1.      Allocated memory vs Virtual memory vs HANA memory pool :

a. Allocated memory(RAM) is amount of memory currently allocated from the operating system (current pool size). This is the memory available to HANA database.

b.Virtual memory = RAM+SWAP space allocated to linux processes by the OS. Right?

c. HANA memory pool: the part of the memory currently allocated by the OS from the preallocated memory quota and can increase up to the global allocation limit This is only about the Physical memory?

d. If used memory is the physical memory in use and is a subset (using memory from memory pool) of the memory pool  Why is the code and stack  shown as a part of HANA used memory and not included in memory pool in the diagrams depicting the same.(page 3).

2.      Resident memory vs Virtual memory:

When (part of) the virtually allocated memory actually needs to be used, it is loaded or mapped to the real, physical memory of the host, and becomes “resident”.

              a. Virtual memory- Swap space=Resident memory is right way of understanding?

              b. Doesn't Resident memory also mean Allocated memory in use? i.e. part of HANA memory pool in use?

3.      Used memory vs Resident memory. 

a.      Resident memory is the physical memory actually in operational use by a process.

b.      Used memory is the total amount of physical memory in use by HANA.

c.      How best can the difference be explained? Both definitions talk about physical memory in use.

d.      Why Resident memory remains high even when the used memory decreases.

4.      How to relate  SAP HANA MEMORY POOL and Virtual memory allocated to HANA.

5.      I observe that during my tests HANA CPU consumption increases due to some reason not very clear to me.

a.      I briefly know that it is due to some background processes like delta merge etc… Any pointers/links on the reason for the same?

b.      How to identify and confirm this?

c.      Is there any way to avoid this during my test execution?

6.      The measurements shown in overview tab

a.      should match with the results of the queries mentioned for used memory measurements in the document like used memory, peak used memory, memory consumed by column/row tables etc.... Right?

a.      I see that used memory displayed in the overview tab doesn’t match with the sum of %MEM in the top command for HANA processes many times. I believe there are other activities going on in HANA which can be monitored in the M_Memory table and other tables?

c.      Is the overview tab the best place to note memory and CPU consumption? I actually want to use a unix command to write the CPU usage and Memory used readings to a text in regular frequency of say 30 minutes to 60 minutes. Reason being we run a few jobs which take like 12-24 hours and I want to automate the monitoring. I believe as far as memory usage is concerned the unix commands don't give a complete picture. CPU consumption measurement is still more in sync with overview tab. Agreed?

7.  HANA    Code and stack comprises of?

8.  Data in the tables are stored in memory or on disk?

a. I believe disk is only used for archiving and logs.

b. By default all data is in-memory? Like, if I create a schema and load some data into it, when I execute the query to get the memory consumed by the table, the result shows physical memory consumed number. Right?

c. Unless we unload the tables or HANA does the same if some tables/data are not in regular use, data won’t move to disk from memory? But then I read somewhere data being on the disk istelf. Which is right?

d. when we talk about HANA moving/unloading some tables based on usage, the data is moved to swap space. Right?

Regards,

Sandeep

Accepted Solutions (1)

Accepted Solutions (1)

lbreddemann
Active Contributor
0 Kudos

Hi Sandeep,

alright - this is a very long list of question and I don't have time to address all of it right now.

Therefore I'll start with just some of the questions.

As the majority of this topic really is not so much about SAP HANA but general Linux memory management, please make sure to educate yourself about this as well.

This paper is an excellent source for this (maybe a bit too detailed for your questions):

What Every Programmer Should Know About Memory http://www.akkadia.org/drepper/cpumemory.pdf

2.      Resident memory vs Virtual memory:

When (part of) the virtually allocated memory actually needs to be used, it is loaded or mapped to the real, physical memory of the host, and becomes “resident”.

              a. Virtual memory- Swap space=Resident memory is right way of understanding?

              b. Doesn't Resident memory also mean Allocated memory in use? i.e. part of HANA memory pool in use?

I think you misinterpret the term "virtual memory" here. For Linux this simply is the size of the address room that the process is currently reaching out to.

Let's say the process accesses some data at memory address 0001, say 10K. These 10K would reside in memory between the addresses 0001 and 0011, OK?

Now, the process loads 20K to address 1000, just because e.g. that's the place in memory where you need to put your graphics data if it should be displayed. Still with me? Good!

Now our virtual memory spans from 0001 till 1020 as we've used addresses within this range.

The virtual memory manager of Linux needs to take care of presenting these memory areas to our process as if we would be the only process in the system and that the data would actually be at these places.

On the other hand, how much memory did we use? 10K + 20K = 30K.

Seen from the other side, these 30K are now not available to any other process anymore.

We've used this resource.

And if you follow this idea of "what did we take away from other processes?" then you see that the resident size needs to include all the memory that is reserved for our process.

This includes of course the executable code of the process itself.

It's process structures like the stack.

And of course the actual data the process works on.

If you want to, the resident size is about resource consumption while virtual size is rather a Linux internal management figure (and much less important since we have 64-Bit address room).

7.  HANA    Code and stack comprises of?

Code = the indexserver executable + all loaded libraries and shared objects.

In Commodore C64 terms: it's what you get into memory when you type in LOAD "file name",8,1

Stack = the information about the current state of processing for the process. What threads are created, what files are open, at which address it the current execution for each thread? Stuff like that.

More about that: http://lmgtfy.com/?q=linux+process+code+and+stack

5.      I observe that during my tests HANA CPU consumption increases due to some reason not very clear to me.

a.      I briefly know that it is due to some background processes like delta merge etc… Any pointers/links on the reason for the same?

b.      How to identify and confirm this?

c.      Is there any way to avoid this during my test execution?

Hmm... this one has nothing to do with memory management at all and sticks out from the other questions.

CPU usage is likely increased by... tatataaa... your usage of SAP HANA.

That's the whole point of SAP HANA that you run a query and our software tries to throw as much parallel CPU time at the task as possible to return the query as fast as possible.

You may want to check the "Threads" overview tab in the SAP HANA studio if the CPU usage stays at a high level for a longer period of time.

Ok, my time's up for today and I guess you already have a bit to read and think about.

- Lars

rama_shankar3
Active Contributor
0 Kudos

Thanks Lars for your inputs and thanks Sandeep for asking these questions. Very informative for HANA in-memory topic.

Rama

Former Member
0 Kudos

Hi Lars,

Thanks for the response. I will come back after looking into it.

Regards,

Sandeep

Answers (0)