ESP - problem with security

[Former Member]

Hi,

I have installed ESP 5.1 v4 and I am getting following errror after kicking off script:

cd /opt/sybase/ESP-5_1/cluster/nodes/node1

$ESP_HOME/bin/esp_server --cluster-node node1.xml

(Location of log: /opt/sybase/ESP-5.1/cluster/nodes/node1/clusted.log:)

Jun 11 2014 16:52:09.416 INFO - SAP Sybase Event Stream Processor Cluster Node 5.1.04.00/20131113.1/SP04 PL00/linux/x86_64/64-bit/OPT/Wed Nov 13 08:29:30 PST 2013
Jun 11 2014 16:52:10.130 FATAL - CODE_700219 | Exception decrypting Security/Password

com.sybase.esp.cluster.impl.CipherService$CipherServiceException: Caught exception decrypting text

        at com.sybase.esp.cluster.impl.CipherService.decrypt(Unknown Source)
        at com.sybase.esp.cluster.impl.CipherService.decryptToString(Unknown Source)
        at com.sybase.esp.cluster.impl.SecurityConfig.configure(Unknown Source)
        at com.sybase.esp.cluster.impl.NodeConfig.configure(Unknown Source)
        at com.sybase.esp.cluster.impl.Node.initialize(Unknown Source)
        at com.sybase.esp.cluster.impl.Node.initialize(Unknown Source)
        at com.sybase.esp.cluster.FactoryNode.factory(Unknown Source)
        at com.sybase.esp.cluster.FactoryNode.main(Unknown Source)
Caused by: com.sybase.esp.system.crypto.CipherException: caught exception decrypting data
        at com.sybase.esp.system.crypto.Cipher.decrypt(Unknown Source)
        ... 8 more
Caused by: javax.crypto.BadPaddingException: Given final block not properly padded
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:811)
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:676)
        at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:313)
        at javax.crypto.Cipher.doFinal(Cipher.java:2087)
        ... 9 more
Jun 11 2014 16:52:10.144 FATAL - CODE_700032 | Security configuration failed
Jun 11 2014 16:52:10.144 FATAL - CODE_700012 | Factory of new node with config file [node1.xml] failed

I have changed ESP to use preconfigured username/password (in csi_local.xml) with no luck.

Any suggestions?

[Former Member · Accepted solution]

Hello,

That means that the password in csi_local.xml was encrypted with a different cipher key than the one in your cluster configuration file (node1.xml).  For example, if you look at your cluster configuration file, you will see a <Cipher> configuration parameter:

         <Csi>

            ...etc...

            <File>${ESP_SHARED}/security/csi_native_unix.xml</File>

            <!--Policy>${ESP_SHARED}/security/policy.xml</Policy-->

        </Csi>

        <Keystore>

            <Type>JKS</Type>

            <File>/work/nstack/ESP51_SP4/ESP-5_1/security/keystore_rsa.jks</File>

            <Password encrypted="true" prompt="false" hide="true" verify="true" query="Please enter keystore password">

dARtLm26lFUbjft5rpLkLlhvKOo00nww9bnKx0DwOHxF0G72

</Password>

            <Algorithm>RSA</Algorithm>

        </Keystore>

        <Cipher>

            <File>${ESP_HOME}/cluster/keys/test-name-1/cluster.key</File>

        </Cipher>

That is used to encrypt/decrypt any encrypted passwords in the file.  You should be able to start at step #5 from the following link:

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01611.0514/doc/html/yel1376688436841...

Thanks,

  Neal

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01611.0514/doc/html/yel1376688436841...

[Former Member · Accepted solution]

Thanks Neil for an answer. I have done that before and it did not work. I am still getting the same error.

I have done clean installation, and it is not working... any other suggestions?

[Former Member · Accepted solution]

Hello,

I gave you the wrong link for modifying the password for the csi_local.xml file.  The correct one is here:

   Enabling the Preconfigured User-Name-Password Option

There is a syntax error in the documentation.  The correct syntax is:

esp_cluster_admin --encrypt_text --encode_text

Please enter text to be encoded:

Please re-enter text to be encoded:

{SHA-256:dOnfXHDzffg=}pTC2SP5MGLbxTFcB9bbpfm2PMNdVS878iT6n2JNT6aI=

I've logged a documentation bug for this:

   765906 - Docs provide wrong syntax for encrypting password in csi_local.xml

Now, I'm still confused.  You said you modified node1.xml to use csi_local.xml.  This would not be a clean installation as you have modified what the installer created.  The error message still means what it says in that it can't decrypt one of the encrypted passwords in the "node1.xml" file.  So did you modify anything else?  Did you change what keytore it points to?  Did you change the encrypted passwords used to access the keystore?

        <Keystore>

            <Type>JKS</Type>

            <File>/work/nstack/ESP51_SP4/ESP-5_1/security/keystore_rsa.jks</File>

            <Password encrypted="true" prompt="false" hide="true" verify="true" query="Please enter keystore password">

dARtLm26lFUbjft5rpLkLlhvKOo00nww9bnKx0DwOHxF0G72

</Password>

            <Algorithm>RSA</Algorithm>

        </Keystore>

If you changed this, did you maintain the original formatting with a new line before and after?

Did you change the cluster's encrypted password?

        <!-- The Name defines the Cluster name.-->

        <!-- All Nodes in a Cluster must have the same Name/Password defined.-->

        <Name>test-name-1</Name>

        <Password encrypted="true" prompt="false" hide="true" verify="true" query="Please enter cluster password">

TiEWD6IeSUju587FRC9omc9GCVmk/8zmxwyo9PznAIYuNNPb

</Password>

Thanks,

  Neal

[Former Member · Accepted solution]

I tryied this as well (I have PDF documentation with those steps).

I haven;t changed encrypted password used to access the keystore, neither cluster's encrypted password.

This is what I did:

1. Kick off installation: ./setup.bin

So I have it installed on the /opt/sybase

2.

Modify node1.xml file:

/opt/sybase/ESP-5_1/cluster/nodes/node1/node1.xml

changes:

- It was "true" before

        <Port ssl="false">19011</Port>

        <AdminPort ssl="false">0</AdminPort>

- Comment out:

<!--<File>${ESP_SHARED}/security/csi_native_unix.xml</File>-->

- Add line:

<File>${ESP_SHARED}/security/csi_local.xml</File>

3. sybase-csi file (as per page 52 from

http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01611.0510/doc/

pdf/admin_guide.pdf)

4. Follow instructions from link provided by you: http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01611.0514/doc/html/tbi1353498879466...

My node1.csi file looks like following:

<Node xmlns:xi="http://www.w3.org/2001/XInclude">


    <!-- The Node Name must be unique throughtout the Cluster. -->
    <Name>node1</Name>


    <!-- The Macros node is optional. -->
    <!-- Values of the form ${macro_name} in nodes identified below will expand to the maco value. -->
    <!-- Macros are loaded before all other config -->
    <Macros>
        <!-- The Macro node has an optional 'type' attribute. -->
        <!-- Allowed 'type' values are "value", "envar", "sysproperty" and "prompt". -->
        <!-- If type is value, the literal value specified (subject to expansion) will be used. -->
        <!-- If type is envar, value for Macro will be pulled from environment valiable defined by Macro value. -->
        <!-- If type is sysproperty, value for Macro will be pulled from Java system property defined by Macro value. -->
        <!-- If type is prompt, the value will be prompted for at startup. Hide and verify options are supported. -->
        <Macro name="ESP_HOME" type="envar">ESP_HOME</Macro>
        <Macro name="ESP_HOSTNAME">esplni01.zit.commerzbank.com</Macro>
        <Macro name="ESP_SHARED">${ESP_HOME}</Macro>
        <Macro name="ESP_STORAGE">${ESP_SHARED}/storage</Macro>
    </Macros>


    <!-- The SystemProperties node is optional. -->
    <SystemProperties>
        <!-- If a Property is defined, that Java system property will be set to the value specified. -->
        <!-- A Property is macro expanded by default. -->
        <!-- To disable expansion, set attribute expand="false". -->
        <!-- prompt="true" will cause the value to be prompted for at startup. Hide and verify options are supported. -->
        <!--
        <Property name="some.system.property.to.set">some value</Property>
        <Property name="some.other.system.property.to.set">some other value</Property>
        -->


        <Property name="esp.home">${ESP_HOME}</Property>
        <Property name="esp.shared">${ESP_SHARED}</Property>


        <!-- If using Kerberos authentication, the following properties-->
        <!--  should be set to their appropriate values. -->
        <Property name="java.security.krb5.realm">REALM_PLACEHOLDER</Property>
        <Property name="java.security.krb5.kdc">KDC_PLACEHOLDER</Property>
    </SystemProperties>


    <!-- A Cluster Node can be a Controller and/or a Manager. It must be at least one. -->
    <!-- In this instance, the Node is both. -->
    <Controller enabled="true">


        <!-- The ApplicationTypes define all application types this Controller Node will be able to launch. -->
        <ApplicationTypes>
            <ApplicationType name="project" enabled="true">
                <Class>com.sybase.esp.cluster.plugins.apptypes.Project</Class>
                <StandardStreamLog enabled="true" />
                <Properties>
                    <Property name="esp-home">${ESP_HOME}</Property>
                    <Property name="hostname">${ESP_HOSTNAME}</Property>
                    <Property name="ld-preload">${ESP_HOME}/lib/jre/lib/amd64/server/libjsig.so</Property>
                    <Property name="services-file">${ESP_HOME}/bin/service.xml</Property>
                    <Property name="base-directory">${ESP_SHARED}/cluster/projects/test-name-1</Property>
                    <Property name="ssl-key-file">${ESP_HOME}/cluster/keys/test-name-1</Property>
                    <Property name="ssl-key-file-encrypted">true</Property>
                </Properties>
            </ApplicationType>
            <ApplicationType name="ha_project" enabled="true">
                <Class>com.sybase.esp.cluster.plugins.apptypes.HaProject</Class>
                <StandardStreamLog enabled="true" />
                <Properties>
                    <Property name="esp-home">${ESP_HOME}</Property>
                    <Property name="hostname">${ESP_HOSTNAME}</Property>
                    <Property name="ld-preload">${ESP_HOME}/lib/jre/lib/amd64/server/libjsig.so</Property>
                    <Property name="services-file">${ESP_HOME}/bin/service.xml</Property>
                    <Property name="base-directory">${ESP_SHARED}/cluster/projects/test-name-1</Property>
                    <Property name="ssl-key-file">${ESP_HOME}/cluster/keys/test-name-1</Property>
                    <Property name="ssl-key-file-encrypted">true</Property>
                </Properties>
            </ApplicationType>
        </ApplicationTypes>
    </Controller>


    <Manager enabled="true">
        <!-- The ApplicationHeartbeatTimeout node is optional -->
        <!-- The first Manager in the Cluster will determine the value Cluster wide -->
        <!-- The value is in milliseconds -->
        <!--
        <ApplicationHeartbeatTimeout>30000</ApplicationHeartbeatTimeout>
        -->
    </Manager>


    <Rpc>
        <!-- The Host node is optional -->
        <!--
        <Host>${ESP_HOSTNAME}</Host>
        -->


        <Port ssl="false">19011</Port>
        <AdminPort ssl="false">0</AdminPort>
    </Rpc>


    <Cache>
        <!-- The Host node is optional.  Only used if Manager node above is enabled. -->
        <!--
        <Host>${ESP_HOSTNAME}</Host>
        -->


        <!-- The Port node is only used if Manager node above is enabled. -->
        <Port>19001</Port>


        <!-- The Name defines the Cluster name.-->
        <!-- All Nodes in a Cluster must have the same Name/Password defined.-->
        <Name>test-name-1</Name>
        <Password encrypted="true" prompt="false" hide="true" verify="true" query="Please enter cluster password">
LH0Jev3YIlMRRXTwfzbqVST1hEMdNAf4YHjphVfTCq/SzNny
</Password>


        <!-- If Multicast is not enabled, the Managers node below must be enabled.-->
        <Multicast enabled="false">
            <Group>224.2.2.7</Group>
            <Port>54323</Port>
        </Multicast>


        <!-- If Multicast is not enabled or the Manager node above is not enabled, -->
        <!-- the Managers node below must be enabled. -->
        <!-- The Managers node specifies all Manager enabled nodes in the Cluster -->
        <Managers enabled="true">
            <Manager>localhost:19001</Manager>
        </Managers>


        <Persistence enabled="false">
            <Directory>${ESP_STORAGE}</Directory>
        </Persistence>
    </Cache>


    <Security>
        <Csi>
            <!-- The File node is macro expanded by default. -->
            <!-- To disable expansion, set attribute expand="false". -->
            <!-- This node specifies the CSI configuration file to use. -->
            <!-- There are currently 6 distributed CSI config examples. -->
            <!-- csi_native_nt.xml uses a CSI supplied LoginModule that provides native OS authentication on Windows. -->
            <!-- csi_native_unix.xml uses a CSI supplied LoginModule that provides native OS authentication on Unix. -->
            <!-- csi_ldap.xml uses a CSI supplied LoginModule that provides LDAP authentication. -->
            <!-- csi_kerberos.xml uses an ESP supplied LoginModule that provides Kerberos authentication. -->
            <!-- csi_rsa.xml uses an ESP supplied LoginModule that provides RSA authentication. -->
            <!-- csi_boe.xml uses an ESP supplied LoginModule that provides SAP BI authentication. -->
            <!--<File>${ESP_SHARED}/security/csi_native_unix.xml</File>-->
            <File>${ESP_SHARED}/security/csi_local.xml</File>
            <!--Policy>${ESP_SHARED}/security/policy.xml</Policy-->
        </Csi>
        <Keystore>
            <Type>JKS</Type>
            <File>/opt/sybase/ESP-5_1/security/keystore_rsa.jks</File>
            <Password encrypted="true" prompt="false" hide="true" verify="true" query="Please enter keystore password">
ywMJ0f2DIvsRlHRsfwfqSySKhGodhQctYHTpUFeuCtDS/tpj
</Password>
            <Algorithm>RSA</Algorithm>
        </Keystore>
        <Cipher>
            <File>${ESP_HOME}/cluster/keys/test-name-1/cluster.key</File>
        </Cipher>
    </Security>
</Node>

Thanks,

Jack

[Former Member · Accepted solution]

Hello,

Thanks for providing the file.  I used your file (modified the host name and path to the keystore_rsa.jks and the csi_local.xml).  When I started ESP, I got the exact same error as you.

I then ran the following twice.  One time for may "keystore_rsa.jks" and one time for the cluster password:

esp_encrypt --encrypt $ESP_HOME/cluster/keys/test-name-1/cluster.key --text

Please enter the value to be encrypted:

Please re-enter the value to be encrypted:

Copy the encrypted value below and paste it into config file.

uH5azn3jsrqstGoBcrSvzAFTA+hStbvkVbPysBHveD7x0Zhn

I was then able to start the cluster.

So if you generate new keys for those two parameters, you still get the same error?

What version of Linux are your running?

Can you provide the output of "ldd $ESP_HOME/bin/esp_encrypt"?

Thanks,

  Neal

[Former Member · Accepted solution]

Hi Neal,

Actually we may have something.

When I run:

-bash-4.1$ ./esp_encrypt --encrypt /opt/sybase/ESP-5_1/security/keystore_rsa.jks --text

I am getting following output:

"Could not create cipher, error=could not parse key"

maybe cipher is wrong?

but when I am running:

-bash-4.1$ ./esp_encrypt --encrypt $ESP_HOME/cluster/keys/test-name-1/cluster.key --text

Please enter the value to be encrypted:

Please re-enter the value to be encrypted:

Copy the encrypted value below and paste it into config file.

qy7qEZ2oFx5T1EpXTdvw6IMvmM5W6jDMNA6/7cx2qgG0rRdT

password is getting generated and same exception occure.

What version of Linux are your running?

Linux esplni01 2.6.32-279.5.1.el6.x86_64 #1 SMP Tue Jul 24 13:57:35 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

Red Hat Enterprise Linux Server release 6.3 (Santiago)

Can you provide the output of "ldd $ESP_HOME/bin/esp_encrypt"?

-bash-4.1$ ldd $ESP_HOME/bin/esp_encrypt

        linux-vdso.so.1 =>  (0x00007fffce6e0000)

        libesp_system_lib.so => /users/taffxmup/sybase/ESP-5_1/bin/../lib/libesp_system_lib.so (0x00007f471b673000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003ec5400000)

        librt.so.1 => /lib64/librt.so.1 (0x0000003ec5800000)

        libicui18n.so.44 => /users/taffxmup/sybase/ESP-5_1/bin/../lib/libicui18n.so.44 (0x00007f471b27e000)

        libicuuc.so.44 => /users/taffxmup/sybase/ESP-5_1/bin/../lib/libicuuc.so.44 (0x00007f471af29000)

        libicudata.so.44 => /users/taffxmup/sybase/ESP-5_1/bin/../lib/libicudata.so.44 (0x00007f4719eec000)

        libstdc++.so.6 => /users/taffxmup/sybase/ESP-5_1/bin/../lib/libstdc++.so.6 (0x00007f4719be7000)

        libm.so.6 => /lib64/libm.so.6 (0x0000003ec6800000)

        libgcc_s.so.1 => /users/taffxmup/sybase/ESP-5_1/bin/../lib/libgcc_s.so.1 (0x00007f47199d1000)

        libc.so.6 => /lib64/libc.so.6 (0x0000003ec5000000)

        /lib64/ld-linux-x86-64.so.2 (0x0000003ec4800000)

        libdl.so.2 => /lib64/libdl.so.2 (0x0000003ec4c00000)

Thanks,

Jack

[Former Member · Accepted solution]

Hello,

You can't use the Java keystore "keystore_rsa.jks" as a cipher key so that first error is expected.

The ldd output shows it is loading libraries from a directory other than the one you installed in:

   /users/taffxmup/sybase/ESP-5_1/bin/

Your configuration file is pointing to a different directory:

  /opt/sybase/ESP-5_1/

Do you have a different version of ESP installed in /users/taffxmup?

   /users/taffxmup/sybase/ESP-5_1/bin/esp_server -v

   /opt/sybase/ESP-5_1/bin/esp_server -v

What is $ESP_HOME pointing to?

Do you have both directories in your LD_LIBRARY_PATH? (env | grep LD_LIBRARY_PATH)

I think something is getting mixed up in your environment causing the error.

Thanks,

  Neal

[Former Member · Accepted solution]

Hello Neal !

Youpii, I get rid of the error- thanks a lot for your helpful input.

So the problem was due to the wrong path (there was an old instance of ESP in the /users/taffxmup so path were pointing to the wrong location, so cipher was wrong.

Solution:

Generate password using instructions from the link (as per message from Neal:17-Jun-2014 17:13)

Enabling the Preconfigured User-Name-Password Option

Hint:

Make sure all environment variables are pointing to your current installation. (My $ESP_HOME directory was pointing correctly but LD_LIBRARY_PATH not. (as per message from Neal: 17-Jun-2014 19:16).

Thansk again for sorting this out.

Thanks,

Jack