Trick(s) to speeding up DI API (specifically DTW)

[Former Member]

Are there any tricks to speed up the DI API and/or DTW (9.0 PL17 64bit)?  I am running on a 2 cpu, 16 GB of RAM.  DTW is using 20 % of CPU and 1.5GB of RAM.  However it is taking over 12 hours to process a file of 37.5K records.  I would like to speed that up to about 4 hours.  Any thoughts?

[former_member201110]

Hi Donald,

Have you tried cutting your source file in to smaller bits (maybe even as small as 1000 each) and importing each one separately? DTW doesn't cope well with larger files. You could also consider running multiple sessions of DTW concurrently (it won't halve the time if you run two sessions but it should significantly reduce the overall time).

What type of data are you importing? In a lot of cases, writing your own code can result in a more efficient solution.

Kind Regards,

Owen