SAVEPOINT Operation in HANA

[former_member652563]

HANA Gurus,

I am struggling with the concept of SAVEPOINT  and Shadow page in HANA.  The question I am trying to ask is:

Does the SAVEPOINT makes the HANA DB consistent.

What I am trying to understand is that what happens to undo data or Shadow Pages which have old data after SAVEPOINT.

According to HANA documentation SAP HANA always perform a SAVEPOINT operation before starting the backup. Therefore, there is no technical need to apply logs after restoring from backups. The SAVEPOINT operation makes the database backups consistent.

I am coming from an Oracle background where you need to apply some logs if you are restoring from an online backup to bring the database in consistent state. Restoring only from online backup in Oracle without applying any logs does not restore Oracle DB to an consistent state.

Thanks in advance for your response

Thanks,

Nasir Syed

[lbreddemann · Accepted solution]

The persistence concept of Oracle and SAP HANA are very different. SAP HANA is a lot more like SAP MaxDB in this regard.

Anyhow, to be restart-able after a shutdown, crash or after a restore, SAP HANA needs to be able to get a consistent version of the database again. Consistent here means that all transactions are either committed or successfully rolled back after the startup/recovery process.

To be able to do that, SAP HANA stores all currently used in-memory data into pages (more or less randomly scattered around in the data files.

It also saves the information about the transactions that were ongoing at the time when the savepoint was started plus the required undo-information.

So, upon restart time, SAP HANA will find any open transaction and, as long as no additional redo logs are supplied for recovery, roll back those transactions. The result is a consistent database.

With Oracle, the data backup is not self-contained in this way, but requires also the redo logs (either backups or online), to get from the inconsistent backups to a consistent database.

That's about the savepoints.

The shadow page concept is more about how to allocate and reuse pages in the data file and doesn't affect the recover-ability that much. But it e.g. frees you (to a very large extent) from the need for data file reorganisation.

Here changed pages will not be overwritten directly, but instead marked as available and the changed content is placed at some other available location in the data file. Since SAP HANA keeps track on which pages contain the current data, there is no need to overwrite or clear unused pages, so after some time the whole data file will contain some data.

- Lars

[former_member652563 · Accepted solution]

Sunil and Lars,

Thanks to both of you for your replies.

Lars,

The following statement from your email finally sums up the whole concept of SAVEPOINT and How restoring only from backups in SAP HANA gets a consistent database.

So, upon restart time, SAP HANA will find any open transaction and, as long as no additional redo logs are supplied for recovery, roll back those transactions. The result is a consistent database.

However, regarding the shadow page this is how I understood the concept, the shadow page keeps the old data and new data is written to the actual physical page which had the original old data. The data from shadow page is there in case of rolling back the transaction. Let me know if my understanding is right here.

If my understanding is right then next question is what happens to the Shadow Page if user commits the transaction.

The second question is how long does the shadow page exists if the user does not commit the transaction and how are they (Shadow Pages) affected by  SAVEPOINT.

Thanks again for all your help,

Nasir Syed

[lbreddemann · Accepted solution]

The shadow page process has actually nothing to do with the transaction handling (commit/rollback) on user session level.

It's really decoupled from that.

SAP HANA keeps undo/redo information in different data structures, but the pages that get written to the data files are really only about the storage management.

Concerning the "lifetime" of a page... whenever a page gets changed in any way, it would be recreated as a new page. Since table data is not directly linked to pages, but internally to data container structures, which in turn will create persistent representations of their payload to be stored in pages, a lot of different things could lead to the change of an individual page.

Anyhow, when you check how many pages of all of the allocated pages get written out during a savepoint you can tell, that under "normal" conditions in a transaction processing oriented systems, only a small part of the pages will be changed.

Sure enough, when massive bulk data loading occurs, this share can easily become quite large and the more pages get changed during a savepoint, the shorter the average "lifetime" of a single page version will be.

- Lars

[former_member652563]

Lars,

Thanks for your response and apologize for responding to your response so late.

The main message I got from your post is that Shadow page has nothing to do with database recover-ability. It has something to do with efficiently organizing pages( basic unit of storage) in data volumes.

Is there ay article or SAP note that explains this concept in simple English.  I got the main message but now I am interested in how this concept helps in better organizing pages in data volumes.

Thanks again for all your help and support,

Nasir Syed

[lbreddemann]

I'm not aware of any piece of SAP HANA documentation that explains this rather well known (https://en.wikipedia.org/wiki/Shadow_paging) to a great extent.

However, maybe this presentation of the SAP MaxDB implementation of that kind of functionality (http://maxdb.sap.com/training/expert_sessions/SAP_MaxDB_ShadowPageAlgorithm.pdf) provides enough clarification.

Just to be clear: this is not the SAP HANA implementation, so several aspects will differ!

- Lars

[Former Member]

Hi Nasir,

HANA Database Savepoint commits the changes in memory to the physical disks and makes it consistent

HANA uses the Shadow paging concept to recover the Data from the previous savepoint version if there is a crash of the system during the savepoint. Although we do not have a written statement on the status of old shadow pages after the savepoint; I beleive after the redo logs are backued up (when log_mode is set to NORMAL) and the savepoint is complete the old shadow pages are discarded

During the recovery from a crash, the most recent Savepoint is read from the data volumes and the redo logs entries written to the log volumes since the last savepoint can be replayed

<<< copyrighted graphic removed by moderator >>>

Hope this is useful

Message was edited by: Lars Breddemann Please don't post copyrighted material without permission and attribution.