OrientDB Running Out of Memory

In our current project in which we use OrientDB (v3.0.19 – standard configuration, mainly) to store process data we are facing the problem that the database keeps running out of memory. And when running out of memory the system does not respond any more (e.g. the Web GUI does not open) and it sometimes corrupts indices or the database. Data is not too big (around 50GB) – in other projects we have already used OrientDB to store by far more data and not running into these issues. But in our current project the single documents are comparably large (1 up to 15MB) which might cause the problems!? We already have switched to manually processing select, insert, delete operations in batches (e.g. of 100 documents).
E.g.:

let cnt = SELECT count(*) as n FROM ProcessDataStg LIMIT 1;
while ($cnt[0].n > 0) {
    let curBatch = SELECT FROM ProcessDataStg LIMIT 100;
    INSERT INTO ProcessData FROM (SELECT FROM $curBatch); 
    DELETE FROM $curBatch; 
    let cnt = SELECT count(*) as n FROM ProcessDataStg LIMIT 1;
}

This helped us but still we are facing OutOfMemory exceptions after some time with the descried effects. And the system should be a data source for our partner’s data scientists and it will not be acceptable if selecting an amount of data which is too big (-> OutOfMemory) will kill the whole system.
Maybe this problem can be tackled by adjusting OrientDB’s configuration (e.g. reduce the size of internal data chunks?)? We already have increased Xmx and Dstorage.diskCache.bufferSize to quite large values (32GB in sum) which solved the problem for the time being. But what if data size grows? Will we then run into the memory issues again?

Hi, it is unlikely but it also depends on the server and JVM settings so you may want to check the docs here:
http://orientdb.com/docs/3.0.x/tuning/Performance-Tuning.html
and here if you’re still having issues after tuning it
http://orientdb.com/docs/3.0.x/troubleshooting/Leak-Detector.html

Thanks a lot for your post. The performance tuning site we already looked at and implemented the suggested points, which seemed to be promising in our case. Activating the leak detector is a very good point - we will give it a try. We ourselves already had the feeling that the issues might be caused by a memory leak…