Adeptia Performance FAQs

Should Backend and Log Database be on different servers?

Yes, especially if you are running large number of process flows every day it is recommended to use Backend Database on one server and Log Database on another server. 

Can there be different type of Database for backend and log? What is the downside of it?

Yes, you can and there is no downside of it.

How to determine Queue process limit and Concurrency settings?

It depends on the size and type of file you process and the system resources (CPU and Memory) you have.

As a general rule, we suggest:

If the expected files to be processed are relative small (say <2MB) then we can set the queue processor concurrency to be 50 if there is 12GB assigned to Kernel Heap size.

Typical formula for concurrent value = Kernel memory / (expected file size * 100)
= 12GB / (2MB x 100) = 12GB/200MB = 60. So, a value of 50 would be relatively safe.

For a scenario where the max file size can be 10MB file, we will have:
= 12GB / (10MB x 100) = 12GB/1000MB = 12. So, a value of 10 would be relatively safe.

This assumes mapping rules that are of medium complexity with no external lookups etc. If the mapping rules are complex then we can half the concurrent value.

If the file format is XML, the multiplier can be lower such as 10x (rather than 100x). But if the file is text or excel or EDI, then the multiplier should be 100x because those files will become 10x bigger when we convert them into intermediate XML files so they need 10x x 10x = 100x memory space to run.

How to determine if multiple DB lookups within a map are not using multiple DB connections against the recommendation to use a single connection?

As long as DB lookups are using same DB Info activity, only one connection is created per mapping activity. 

What does 0 or -1 indicate for “Maximum number of files that will be processed by an event in each polling frequency”?

0 or -1 indicates that there is no limit on number of files to be processed by event in each polling. 

What is event Metadata?

Event metadata contains name, size, and last modified time of a file. This information is used to determine whether the file has been already processed or need to be processed by the event.  Event metadata doesn’t contain the file data copy.

What does “Max Threads” property in jetty.xml determine?

Max thread determines the number of concurrent web requests (GUI + Web Service calls) that are allowed to run. We can fine tune this number based on resources (Memory and CPU) available for WebRunner.

Is there an upper limit to org.quartz.threadPool.threadCount

Its value depends on the number of concurrent jobs (events) to be executed.  This is what Quartz documentation recommends:

Can be any positive integer, although you should realize that only numbers between 1 and 100 are very practical. This is the number of threads that are available for concurrent execution of jobs. If you only have a few jobs that fire a few times a day, then 1 thread is plenty! If you have tens of thousands of jobs, with many firing every minute, then you probably want a thread count more like 50 or 100 (this highly depends on the nature of the work that your jobs perform, and your systems resources!). 

How to view CPU and memory usage?

CPU, memory usage and other system statistics can be viewed from system console. To view system console login as Administrator. Go to Administer tab and select System Console.

What are the recommendations for database loading?

- Use database (stored procedures, queries, joins) for in-database processing whenever that option is available rather than extracting large amount of data from the database and processing in Adeptia
- Use Database loader programs for faster loading of bulk data into databases rather than using Adeptia JDBC connectors

Where database intensive operation need to be performed, we recommend bulk loading where that feature is available. Instead of using Adeptia JDBC connectors, use database feature instead.

For example, if you have to insert huge data in the database, there are two options:

1. Use Adeptia Suite's Database or Advanced Database target to insert data into the DB table. 
2. Do the conversion of source data using Adeptia Suite and finally insert it into the database using the database specific program because the database specific program will load the data faster and will not overload Adeptia Suite application.
   
   

What does it mean: "If the RAM size assigned to Adeptia Kernel JVM is 8GB then use half of that for the transaction processing and that to be based on the maximum size of data volume"?

As a general rule, we recommend user to define Queue Processor concurrency, so that half to Kernel JVM at max should be used for data processing.