PowerDesigner for Hive/HBase

[former_member1144245]

Wondering if anyone is using PowerDesigner for modeling tables for Hive and/or HBase? I have started a custom .xdb file for Hive but it falls short when dealing with complex data types and there are additional customizations needed that I have not figured out yet. I've heard that the next version will have some sort of support for big data, but no idea what that will look like.

[marc_ledier · Accepted solution]

Hi Monique,

We plan to include a new xdb for 16.5 SP04.

I would be interested in your specific request about Hive in PowerDesigner.

You can write directly to me (marc.ledier@sap.com).

Regards,

Marc

[Former Member · Accepted solution]

Hi Marc,

do you know when it will be released? I'm interested in it.

Thanks

Karel

[Former Member · Accepted solution]

Any updates on Hive support?

I don't see anything in the notes for 16.5 SP04 about this.

I'd also be interested in any future plans to support modeling JSON, and I'm getting questions from our modeling team about being able to model other NoSQL structures.

[marc_ledier · Accepted solution]

Hi Eric,

In the SP04 you should be able to create a physical data model targetting the "Hadoop Hive 1.0" xdb.

Marc

[Former Member · Accepted solution]

Hi Eric,

Marc has answered about our support for Hive.

In looking at support for modeling non-SQL databases, I understand being able to model JSON, however, the challenge is when looking at databases like Mongo and others that use a JSON or JSON-like syntax, is that the database structure relies on the document to self-describe the content, and there's no declaration of the content structure in a catalog or similar structure.

While we are researching, I am interested in your take on how we could model such things, like a statistically reasonable sample size from documents for common traits.  And, what is the value of such modeling - to be able to construct documents or to be able to assign additional metadata for quality, accuracy or some other stewardship?

Any ideas you have to share is greatly appreciated.  The concept of modeling for Big Data is high on our priority list, and some of these things are still being discussed both internally and externally with many customers that are working with us to come up with the right specifications.

However as you can see, we are already supporting the structured formats well with the latest release, which is in our view the right first step.

- David.

[Former Member · Accepted solution]

Hi,

I want to use PowerDesigner for modeling tables for Hive. What are the PowerDesigner components i need to install on my machine ?

Thanks,

Bob.

[Xiao-yun · Accepted solution]

You just need PowerDesigner 16.5 SP04.

There is a new DBMS: Hadoop Hive 1.0.

[Former Member · Accepted solution]

Hi Xiao-Yun

I'm looking like crazy over the SAP Web Site to find the download.

Do you have by any luck a direct (URL) link to PowerDesigner 16.5 SP04 download Page ?

Thanks,

Bob.

[Former Member · Accepted solution]

1) Enter service.sap.com website.

2) Enter Software Downloads > Support Packages and Patches > A-Z Index

3) Go to letter P and choose SYBASE POWERDESIGNER/SAP PowerDesigner on the list

4) You may find two alternatives:

- SYBASE POWERDESIGNER 16.5/SAP PowerDesigner 16.5 <--- Choose this one

- SYBASE POWERDESIGNER SE 16.5/SAP PowerDesigner, standard edition 16.5

5) Choose SYBASE POWERDESIGNER 16.5 -> Windows on x64 64bit

6) Download and install EBF 16.5 SP04 PL01 PowerDesigner (x64)**

** You can confirm if your PowerDesigner installation is x64 by checking whether PowerDesigner is installed at C:\Program Files\Sybase\PowerDesigner that is installed in "Program Files (x86)" directory, that means it would be 32-bit, and then corresponding patch to download would be the corresponding "EBF : 16.5 SP04 PL01 PowerDesigner (w32)" through step 5 "SYBASE POWERDESIGNER 16.5 > Win32" instead.

7) After installing the Patch level, enter PowerDesigner again and confirm if it now shows: 16.5 SP04 PL01 at Menu Help > About PowerDesigner

Note: If you are using PowerAMC instead, the steps are the same. Just follow the very same steps above, but by changing Product Name to SYBASE

POWERAMC/SAP PowerAMC in step 3 and by choosing the corresponding EBF numbers for PowerAMC instead.

For more information go to :

[Former Member]

Hi David

We are building HBASE solutions as well as Oracle solutions. We have stored all our Oracle data models in PowerDesigner and would like to do the same for the HBASE solution.

We would also want to reverse engineer the tables/schemas into PowerDesigner to keep the model aligned.

Can you please advise if 1) it is possible to model HBASE tables and 2) if the tables can be reverse engineered?

Chantel

[Former Member]

I haven't had a chance to try this yet, Monique, but it is something I'd be very interested in also.

Would it be possible for you to share the .xdb file you've started so that I can better understand the error you're getting, and try to help you get it working?

[former_member1144245]

Hello Eric. Not getting any errors. There is just a lot needed before PowerDesigner can support modeling for Hive. Some things I was able to customize myself - e.g. correct list of data types for Hive, certain statements desired before or as part of the create table DDL. Other things are more complex e.g. optional parameters for create table DDL, which I have been able to customize but which still needs manual effort to provide appropriate variables. The most difficult is what to for complex data types e.g. Struct - which is effectively a nested structure within the table. PowerDesigner is not (and may never be) set up for that. Not even sure what that would look like. It may be that PowerDesigner can only represent the logical model but not the physical. Don't see a way to attach the .xdb I've been working on so let me know how I can send it.