Number of AL_ENGINE Processes ?

[Former Member]

Hi All,

Can i know on what basis number of AL_ENGINES processes will be running in DS Server? I have 4 job severs and 1 Access server in DS server and AL_ENGINE processes will be running around 20 and consuming more memory. No jobs are running at the moment.

Thanks 

[former_member187605 · Accepted solution]

The maximum number of al_engine processes is determined by a setting in the DSCONFIG.TXT file, MAX_NO_OF_PROCESSES, default value is 8.

If no jobs are running, you should only see 1 al_engine process per server.

[Former Member · Accepted solution]

Thanks a lot Dirk for your response !

i have 11 real time services running and some of them have more than one instance, so total number of instances are 21 for these real time services and i have 22 AL_ENGINE processes are running on Server. so that means these 21 real time services instances created 21 AL_ENGINES and 1 for job server, is that correct ?

[former_member187605 · Accepted solution]

That sounds logical.

[Former Member · Accepted solution]

that's correct, i just shut down on real time service and it has 4 instances a\ and AL_ENGINE processes came down to 18.

Thanks again Dirk !

[former_member213365]

Maximum Number of Engine Processes

This setting limits the number of al_engine processes that can be spawned by a job. This is a per-job value, not an overall job server processes value. This means that if ten jobs are running at the same time and the value is set to 8 (the default) then there is the possibility of 80 al_engine processes. Jobs that run many parallel Dataflows could be impacted. A Workflow doesn't spawn an al_engine process but a Dataflow does. The job itself takes up one of the al_engine processes. The Workflow shown in below executes eighteen child Workflows in parallel.

Each of the 18 Workflows contains at least one Dataflow. Only seven of the Dataflows could run at one time if the Maximum Number of Engine Processes setting is using the default value of eight.  Seven Dataflow al_engine processes plus one Job al_engine process equals eight.  Dataflows number eight and higher will queue up.  As soon as one of the first seven Dataflows completes then the next Dataflow in the queue will run.

For this particular Workflow a value higher than the default would be beneficial. As long as the eighteen Dataflows do not swamp the database servers or use large amounts of memory on the job server there shouldn't be a problem.

Some Dataflows execute queries that take longer to return a result set. Configuring a larger number of al_engine processes for execution would allow Dataflows with quick responding result sets to begin and perhaps finish processing before the longer running Dataflows have received the first row of their result set. However, it is unlikely that only one job is running. If a worst case assumption is made that all of the Dataflows within the eighteen Workflows use the maximum amount of memory (about 4 GB on a 64 bit O/S) and the Maximum Number of Engine Processes setting is 19 then there could be an attempt made to use almost 80 GB of RAM just for the Dataflows. This would likely bring the Job Server operating system to its knees.

A reasonable compromise needs to be made to balance individual job execution against overall job server performance.

The Maximum Number of Engine Processes setting is dynamic. The next time a job is executed it will pick up the setting. The job server does not need to be restarted to make the setting effective. The setting can be changed by opening Data Services Designer and going to Tools/Options/Job Server/Environment. Designer must have a valid connection to the job server. The setting can also be changed by editing the DSConfig.txt file. The line to edit is MAX_NO_OF_PROCESSES found in the AL_Engine section.