Hi,

there seems to be a memory leak in the OpenFlight API: mgCloseDb does not release all memory which mgOpenDb has requested before.

I’m currently using the API to do the same operations on a high number of files. That means I’m using mgOpenDb and mgCloseDb a lot. After some 1000 files my application crashes because the system is out of memory. I realized while searching for the error, that these two functions seem to leak (without doing anything else than opening and closing databases the memory usage increases constantly).

int main (int argc, char *argv[])
{
   mgInit(&argc;, argb);
   mgrec* db = NULL;
   
std::set<std::string> files; 

// [..] read in files

for(int i = 0; i < numFiles; ++i)
{
   db = mgOpenDb( files.c_str() );
   if(db)
       mgCloseDb(db);
   db = NULL;
}

   mgExit();
}

I confirmed this behaviour with OpenFlight API version 3.2, 3.5 and 4.0 (using Windows XP and Microsoft Visual Studio 8; tested 3.5 also on Linux). I also tried models without any colors, materials, textures, etc. assigned, but the effect remains. Admittedly the amount of memory not freed gets smaller with every call to open/close. If you open and close the same file over and over, after a while no further memory is lost.

Any suggestions? Help or an explanation would be much appreciated.

Kind regards,

Katharina

With some further investigation and the help of the support team came the following findings:

There is no leaking memory in case of files without textures and external references. So the problem has to do with OpenFlight APIs handling of textures and external references. Some old forum post on this topic:

In either case, the API may appear to hang onto memory even after all is supposedly freed. This happens due to the way it uses the SmartHeap memory pool mechanism. All of our memory allocation is done via pools. The main advantages of using pools are faster allocations and less fragmentation. The main disadvantage is that even after memory is "freed", it is really not given back to the process heap. That's because the memory pool manager in SmartHeap keeps a free list of memory ready to be reallocated. I don't want to get into a big debate on the merits of memory allocation schemes, just wanted to give you some insight into how Creator and the API do things in this area.

Anyway, this may be why memory appears to be leaking. A good check for this is to open and close the same database a number of times in one process and watch the leaks. If the amount of "unfreed" memory grows a large amount the first time you open-close and only grows a small amount the 2nd and subsequent times, you are seeing SmartHeap memory pooling in action. The small increments you see after the first time is likely to be due to fragmentation. If you find that "unfreed" memory steadily increases each time you open and close a file (increases the same amount 1st and 2nd times), I would look at texture as the prime suspect.

Some other notes on texel management inside Creator and the API:

Like I said above, if your app never tries to access texels of a texture, the API never "loads" the texels from disk into memory. This makes apps that are working on geometry-only run much faster (reading textures you never access unnecessarily wastes time and memory). If your app causes the API to load a texture from disk, the API will keep a copy of it around "forever". This may be where your leaks are. That is, even after you close the db that references a pattern, the API keeps a copy of it in an internal cache. The API (and Creator) keeps a cache of all textures read so the 2nd and subsequent references to a pattern are quicker (at the expense of potentially stale textures lying around internally).

Creator manages this by giving the user explicit control of when the texture cache is emptied. See the Texture Panel on the Preference window.

The API also gives you this control via an undocumented fuction, mgPurgeTextures whose signature is: void mgPurgeTextures(void);

This function is undocumented because it is currently implemented in an unsafe manner. It simply deletes all textures in the cache regardless of whether they are referenced or not. Bad API!! If you want to call it, do so only when no databases are currently in memory (that is when all db's have been closed).

Another explanation:

There are two caches in the OpenFlight API, one for textures and one for external references. These two caches behave differently.

The texture cache is used to store textures referenced by OpenFlight files. When you open an OpenFlight file, all included textures are put into the texture cache. When the OpenFlight file is closed the textures remain in the texture cache. The cache must be emptied manually, or will be automatically emptied when you call mgExit. Emptying it manually is accomplished by calling the mgPurgeTextures() function. This function is not documented because it can be unsafe if used incorrectly. However, you can call mgPurgeTextures() from your OFAPI program. It is not included in any of the header files so the compiler will issue a complaint – however this function is defined in mgapilib.lib so your linker WILL be able to find it and build your program. You can ignore the compiler warning.

If you like, you could declare the function in your code. This will make the compiler happy and the compiler will not issue any warnings. If you want to do this, simply add this line in your code:

void mgPurgeTextures(void);

The next subject is the external reference cache. When you open an OpenFlight file, all external references are put into the external reference cache. Then the OpenFlight file is closed the external reference cache is purged. This is different from the texture cache. Because the external reference cache is always purged when you call mgCloseDb, there is no function to manually purge this cache. You do not need to purge the external reference cache.

Anyway, even if using mgPurgeTextures there is some memory leaking.

For now I have a nasty workaround, but it would be great, if this bug gets fixed in one of the next releases.

Cheers,

Katharina

^I'm glad there's no memory leak of some sort. I was kind of worried about it. Thanks for information and for telling us about it here.

Regards,

Czarny

Simulation pret immobilier

When I try to use the mgPurgeTexures(), I get linking errors...

C++ Builder 2007, converted COFF libs to OMF.

Im also getting the same error..Any suggestions or options we could use or try? Thanks!

Regards,

marykeens

[URL=http://prettravaux.net/pret-travaux/des-pret-travaux-pour-tous-les-gouts]Prêt travaux[/URL]

First, remember mgPurgeTextures is not a fully supported function... use it at your discretion given the info in the many posts about it...

Second... since it is not supported, it is not declared in any of the API header files. If you are trying to call it from a c plus plus file you must declare it correctly so the compiler does not "mangle" the name.

Put the following code in the file where you are calling this function somewhere before you call it.

#ifdef __cplusplus
extern "C" {
#endif

extern void mgPurgeTextures (void);

#ifdef __cplusplus
}
#endif

If that does not work try:

MGAPIFUNC(void) mgPurgeTextures (void);

to declare it inside the extern "C" - just make sure you include mgapiall.h to get the MGAPIFUNC macro defined.

^ It works for me now. Thanks for the help.

Regards,

Debbie

Ordinateur portable pas cher