Hello,

Today, I received a question today about how to compute the maximum memory of a data mining algorithm like FPGrowth in Java.

It is very simple. I do like that.

I add a variable :

double maxMemory = 0;

Then I copy and paste this method:

private void checkMemory() {
        double currentMemory = ((double) ((double) (Runtime.getRuntime()
                .totalMemory() / 1024) / 1024))
                - ((double) ((double) (Runtime.getRuntime().freeMemory() / 1024) / 1024));
        if (currentMemory > maxMemory) {
            maxMemory = currentMemory;
        }
    }

Then, I call the method

checkMemory()

from where I want to check the memory.

Finally, I print the result:

System.out.println("memory : " + maxMemory);

Philippe



Edited 2 time(s). Last edit at 05/08/2012 05:39PM by webmasterphilfv.

one question i have in my mind sir.
please let me clear

this memory show that how much RAM used by JVM?

And if it is RAM than How it show that max memory is used by program because if program execute all the steps then RAM will be realesed.....

Thats true.

I measure max. memory little bit different.
Before starting mining I take memory snapshot. So I got variable memoryBeforeRun.
Then Iam getting snapshot regularly during mining every x ms. So i got max. memory usage.

Finally max. memory is equal to: memoryBeforeRun - maxMemory.

My implemenation in C#:

public class MemoryCrawler
{
/////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////

private static long _maxMemoryInBytes = 0;
private static long _memoryBeforeRunInBytes = 0;
private static readonly object _mutex = new object();
private readonly Timer _memoryCrawler = new Timer(500);

/////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////

public MemoryCrawler()
{
_memoryCrawler.AutoReset = true;
_memoryCrawler.Elapsed += MemoryCrawlerOnElapsed;
}

/////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////

private static void MemoryCrawlerOnElapsed(object sender, ElapsedEventArgs elapsedEventArgs)
{
lock (_mutex)
{
long actualMemory = GC.GetTotalMemory(false);
_maxMemoryInBytes = actualMemory > _maxMemoryInBytes ? actualMemory : _maxMemoryInBytes;
}
}

/////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////

public void Start()
{
_maxMemoryInBytes = 0;
_memoryBeforeRunInBytes = GC.GetTotalMemory(true);

_memoryCrawler.Start();
}

public void Stop()
{
_memoryCrawler.Stop();
}

public long GetMaxMemoryInMBytes()
{
return (_maxMemoryInBytes - _memoryBeforeRunInBytes) / 1048576;
}

/////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////

}

Hello Dvijesh,

>>>> this memory show that how much RAM used by JVM?

The method checkMemory() is for checking how much memory is used by the program running inside the JVM at a particular moment.

>>> And if it is RAM than How it show that max memory is used by program because if program execute all the steps then RAM will be realesed.....

To know what is the max memory, the trick is, as tisonet said, to check the memory several times during the execution. So in my code, it means to call checkMemory() several time during the execution.

For example, consider the PrefixSpan algorithm. You could call the method checkMemory() after each database projection, for example. Then, when the algorithm terminates the variable maxMemory will contain the maximum memory used. It may not be 100 % exact. But it will be very close to the real value.

A better way may be to do as tisonet said and to check the memory before the algorithm starts check regularly an

Beside that, I have one last thing to say about the memory. You probably know that Java use a garbage collector to free some memory when all the memory is used. The garbage collector is "lazy". That means that it will only free memory when all the memory has been used. For example, if you have 1 GB of memory, the garbage collector will start to clean when the 1 GB is full. Why i'm talking about this? It is that if your program reach the point where the garbage collector need to free some memory, it will start to slow down your algorithm and if you are measuring the speed of your algorithm, it will give you some wrong measurements. So, I just want to say to be careful if you reach the limit of memory available on your computer because the garbage collector may interfere with your measurements.

Philippe



Edited 4 time(s). Last edit at 05/08/2012 05:39PM by webmasterphilfv.

Thanks you both

i got so many things now and clear with it.

yes philippe sir i also feel that algorithm run slow when memory reach at the highest point. like you mention 1 GB