Start a new topic

rudimentary file parser with mgWalk?

Greetings,


I believe I can edit files to keep or remove nodes based on the type of records encountered during mgWalk.


Is there an API attribute or whatnot that tracks the hierarchy as mgWalk progresses, so that if there is structure on top of the node being removed, and the node is the only child, then the entire branch can be removed instead of just the node at the end of that branch?


Or is that something I have to do myself, and if so, how would I go about doing that?

In a python list I keep track of elements by their position; how would I do this for a flt file hierarchy??


Thx


Shawn


Hi Shawn, When walking a hierarchy each node is still linked up into the hierarchy. So mgGetParent, mgGetChild, mgGetnext etc.. all still function. Have you tried using the post walk function? This function will be called for a node after all of its children were already called. So the logic goes, if you visit a child in post walk and delete it, when you visit the parent of that child it will be empty. You can check this and delete if empty.

Hi Chris,


Yeah, you had a reply on the forum that mentioned that so I was able to get a little further.  


Can you explain how you move up and down the tree?  So if I go all the way down the left side, how do I go 'back up' to go down any other branch??


Thanks!

manually traversing the tree is simple, but the book-keeping is complicated. For many node, you can go up, down, or to a sibling. Simple see lol, but that is where the simplicity ends. Keeping track of where you are in the tree gets complicated. I don't think you need to go through all those lengths though for your purpose. If all you want to do is remove nodes that have been made empty during a delete pass, simply use mgWalk with a post-walk function. post walk function if rec is one to delete, delete it here if rec is a group (or any node type you don't want to remain if empty) and is currently empty, delete it here If a node is visited in the post walk, you can be assured that all of it's children have been visited and deleted if it was going to be deleted. So in the post walk just see if the node has any children, if none, delete it. (as long it is the node type you want to delete if empty)

OK, thanks.  I'll give that a go.

I'm working on a way to prune everything in a db that is not part of a switch tree.  I need to keep the db root, obviously.


I'm thinking of traversing the db once, identify nodes that are not children of switches, remove them.

Then walk the db again, and if there are no children, delete that node. This would have to happen recursively, I think.


That can't be the best way to do this, right?  What's a better way?


Thx

This feels pretty kludgy, but it also almost works.  If a node meets certain criteria, it gets added to the keepList.  I would like to put nodes not in keepList into a pruneList, but my attempts to do that in-line don't work.  I may need another pass through the database, then I can check against keepList and if not in there, add to pruneList.  

  

	groupCode = 309
	rootCode = 310
	lodCode = 324
	switchCode = 362
	code = mgGetCode (rec)
	name = mgGetName (rec)
	parent = mgGetParent (rec)
	parentName = mgGetName (parent)
	parentCode = mgGetCode (parent)

	no_switches_found = 0

	codeSupported = MG_TRUE
	checkTransforms = MG_TRUE

	# print node name and type  
	# PrintNodeNameAndCode (rec)

	if code == fltSwitch:
		print ('======================')
		print ('parent:',parentName)

		if not parentName in keepList:
			keepList.append(parentName)
		if not name in keepList:
			keepList.append(name)

		# print ('mgDelete (rec)')
	elif code == fltObject:
		if not parentCode == switchCode:
			print ('=================')
			print ('parent:',parentName)
			# print (mgGetCode(parent))
			print ('found object',name)
		
		else:
			if not parentName in keepList:
				pruneList.append(name)
			
			# mgDelete (rec)
	elif code == fltGroup:
		if parentCode == rootCode:
			print ('checking root:', name)

			if not name in keepList:
				keepList.append(name)

		elif not parentCode == switchCode:
			print ('=================')
			#print ('group code:',mgGetCode(rec))
			print ('parent:',parentName)
			print ('parent code:',parentCode)
			print ('found group',name)

			if parentName in keepList:
				numChildNodes = mgCountChild(rec)

				if numChildNodes == 1:
					if mgGetCode(mgGetChild(rec)) == groupCode or mgGetCode(mgGetChild(rec)) == lodCode:
						if not name in keepList:
							keepList.append(name)
					
				else:
					for i in range(1,numChildNodes):
						childnth = mgGetChildNth(rec,i)
						if not mgGetCode(childnth) == switchCode:
							print ('-------------- no switches found!')
							no_switches_found = 1
						elif parentName in keepList:
							if not name in keepList:
								if not no_switches_found == 1:
									keepList.append(name)
		# if we aren't keeping the node then add to prune
		else:
			if not parentName in keepList or parentCode == lodCode:
			
				pruneList.append(name)
				# mgDelete (rec)

	elif code == fltLod:
		print ('parent:',parentName,'LOD code:',mgGetCode(rec))
		numChildNodes = mgCountChild(rec)

		print ('keeping:',keepList)

		if parentCode == lodCode:
			if not name in keepList:
				keepList.append(name)

		elif numChildNodes > 0:
			for i in range(1,numChildNodes):
				childnth = mgGetChildNth(rec,i)
				print ('child:',mgGetName(childnth))
				if mgGetCode(childnth) == switchCode or mgGetName(childnth) in keepList:
					if not name in keepList:
						keepList.append(name)
				else:
					no_switches_found = 1
			if parentName in keepList:
				if not no_switches_found == 1:
					if not name in keepList:
						keepList.append(name)
		print ('found LOD',name)
		print ('=================')

  Here's what I get - I think it's accurate, but this is a simple file:


== keep these nodes ==

res_90

g160

LOD_top

details

signs

s_env_pkdVeh_res_90_0x_0y_0r

s_env_garage_res_90_0x_0y_0r

s_spdLmt_r1b_res_90_0x_0y_0r

s_spdLmt_r1a_res_90_0x_0y_0r



Huh.  A significantly more complex file doesn't work much :(


It keeps all the switches but not the hierarchy as intended.

Hey Shawn, sorry, I thought I replied yesterday but don't see my msg. 


I threw something together and tested it on a moderately complex file.

  • walk to get all the switch nodes into a list. This is used later
  • Then walk again with a post and a pre func.
  • In the pre func, you just keep track of how many switch nodes are above you in the walk
  • In the post func you check to see if you are under a switch. If so you can't delete
  • Then you check to see if you are above a switch. If so you can't delete.
  • Knowing if you are above a switch is the hard part as walking down a tree is complicated. That is where the switch list comes in. It is easy to walk up a tree as everything narrows rather than widens. Basically you just walk up the tree of each switch node in the list. If any of them are a child of the node you are testing, you will find the node while iterating though parents.


Here is the code:


db = mgGetCurrentDb()

def gatherSwitchNodes (db, parent, rec, switchList):
   if mgGetCode (rec) == fltSwitch:
      switchList.append(rec)
   return MG_TRUE

switchList = []
mgWalk(db, gatherSwitchNodes, None, switchList, MWALK_NOREADONLY)

def isNodeParent (parentNode, testChildren):
   for possibleChild in testChildren:
      testNode = possibleChild
      while testNode and testNode != db:
         if testNode == parentNode:
            return MG_TRUE   
         testNode = mgGetParent(testNode)
   return MG_FALSE
   
switchDepth = 0
def checkSwitch (db, parent, rec, data):
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth + 1
   return MG_TRUE
   

def deleteNonSwitchNodes (db, parent, rec, deleteList):
   global switchList
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth -1
      
   # Switch depth tells us if we are under a switch
   # Still need to see if we have a switch as a child
   elif mgGetCode(rec) == fltGroup:
      if switchDepth < 1 and not isNodeParent(rec, switchList) and not rec == db:
         deleteList.append(rec)
   return MG_TRUE
   
deleteList = []
mgWalk (db, checkSwitch, deleteNonSwitchNodes, deleteList, MWALK_NOREADONLY)

print len (deleteList)
for d in deleteList:

 

Awesome, thanks!  This almost works.  I'm looking at the left side of the tree and there's an object that isn't in the deleteList.  Maybe object codes are not supported in this version (I see others that should be OK to delete, I will upload the file)?


Here's my version:


 

from mgapilib import *
import sys


mgInit (None, None)


print("\nOpening database: ",sys.argv[1],"\n")
db = mgOpenDb (sys.argv[1])
if db == None:
   print(mgGetLastError (), "\n")
   mgExit ()

# db = mgGetCurrentDb()

def gatherSwitchNodes (db, parent, rec, switchList):
   if mgGetCode (rec) == fltSwitch:
      switchList.append(rec)
   return MG_TRUE

switchList = []
mgWalk(db, gatherSwitchNodes, None, switchList, MWALK_NOREADONLY)

def isNodeParent (parentNode, testChildren):
   for possibleChild in testChildren:
      testNode = possibleChild
      while testNode and testNode != db:
         if testNode == parentNode:
            return MG_TRUE   
         testNode = mgGetParent(testNode)
   return MG_FALSE
   
switchDepth = 0
def checkSwitch (db, parent, rec, data):
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth + 1
   return MG_TRUE
   

def deleteNonSwitchNodes (db, parent, rec, deleteList):
   global switchList
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth -1
      
   # Switch depth tells us if we are under a switch
   # Still need to see if we have a switch as a child
   elif mgGetCode(rec) == fltGroup:
      if switchDepth < 1 and not isNodeParent(rec, switchList) and not rec == db:
         deleteList.append(mgGetName(rec))
   return MG_TRUE
   
deleteList = []
mgWalk (db, checkSwitch, deleteNonSwitchNodes, deleteList, MWALK_NOREADONLY)

ok = mgCloseDb (db)
if ok == MG_FALSE:
   print("Error closing database\n")


print (len (deleteList))
for d in deleteList:
   print (d)

 


flt
(864 KB)
flt

here's a pic

image

Ah yes, I specifically only deleted group nodes in the script. This can be modified to delete groups and objects. Or all nodes for that matter.


I worry about using a deletelist when having nodes added to the list that could be children of other nodes in the list. dont want to delete a parent then delete a child of a deleted parent. So I just deleted in the postwalk in the one below and added objects to the type to delete

 

db = mgGetCurrentDb()

def gatherSwitchNodes (db, parent, rec, switchList):
   if mgGetCode (rec) == fltSwitch:
      switchList.append(rec)
   return MG_TRUE

switchList = []
mgWalk(db, gatherSwitchNodes, None, switchList, MWALK_NOREADONLY)

def isNodeParent (parentNode, testChildren):
   for possibleChild in testChildren:
      testNode = possibleChild
      while testNode and testNode != db:
         if testNode == parentNode:
            return MG_TRUE   
         testNode = mgGetParent(testNode)
   return MG_FALSE
   
switchDepth = 0
def checkSwitch (db, parent, rec, data):
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth + 1
   return MG_TRUE
   

def deleteNonSwitchNodes (db, parent, rec, data):
   global switchList
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth -1
      
   # Switch depth tells us if we are under a switch
   # Still need to see if we have a switch as a child
   elif mgGetCode(rec) == fltGroup or mgGetCode(rec) == fltObject:
      if switchDepth < 1 and not isNodeParent(rec, switchList) and not rec == db:
         mgDelete (rec)
   return MG_TRUE
   
mgWalk (db, checkSwitch, deleteNonSwitchNodes, None, MWALK_NOREADONLY)

 

I hear you! Thanks!


Hm.  I managed to break it anyways.  Python crashes!


  

from mgapilib import *
import sys


mgInit (None, None)


print("\nOpening database: ",sys.argv[1],"\n")
db = mgOpenDb (sys.argv[1])
if db == None:
   print(mgGetLastError (), "\n")
   mgExit ()

# db = mgGetCurrentDb()

def gatherSwitchNodes (db, parent, rec, switchList):
   if mgGetCode (rec) == fltSwitch:
      switchNameList.append(mgGetName(rec))
      switchList.append(rec)
   return MG_TRUE

switchList,switchNameList = [],[]

mgWalk(db, gatherSwitchNodes, None, switchList, MWALK_NOREADONLY)

def isNodeParent (parentNode, testChildren):
   for possibleChild in testChildren:
      testNode = possibleChild
      while testNode and testNode != db:
         if testNode == parentNode:
            return MG_TRUE   
         testNode = mgGetParent(testNode)
   return MG_FALSE
   
switchDepth = 0
def checkSwitch (db, parent, rec, data):
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth + 1
   return MG_TRUE
   

def deleteNonSwitchNodes (db, parent, rec, data):
   global switchList
   global switchNameList
   global switchDepth
   if mgGetCode (rec) == fltSwitch:
      switchDepth = switchDepth -1
      
   # Switch depth tells us if we are under a switch
   # Still need to see if we have a switch as a child
   elif mgGetCode(rec) == fltGroup or mgGetCode(rec) == fltObject:
      if switchDepth < 1 and not isNodeParent(rec, switchList) and not rec == db:
         mgDelete (rec)

   return MG_TRUE
   
mgWalk (db, checkSwitch, deleteNonSwitchNodes, None, MWALK_NOREADONLY)

ok = mgWriteDb (db)
if ok == MG_FALSE:
   print("### Error writing to database!\n")

ok = mgCloseDb (db)
if ok == MG_FALSE:
   print("Error closing database\n")

mgExit()

  

C:\Presagis\Suite22\OpenFlight_API_22_0\samples\scripts>python parser2.py ablip_complex.flt

I: OpenFlight API version 22.0.0.

I: Loading plugin <OpenFlight Data Dictionary> from <C:/Presagis/Suite22/OpenFlight_API_22_0/bin_x64/release/fltdata.dll>...

I: Site <FLTDATA> registered for plugin <OpenFlight Data Dictionary>.

I: Plugin <OpenFlight Data Dictionary> loaded.

Opening database: ablip_complex.flt


C:\Presagis\Suite22\OpenFlight_API_22_0\samples\scripts>python --version

Python 3.7.6


 

 

 

On the plus side, it does work for my simple case. Just not the complex one.


image


Oops.  It also crashes Creator :(


image


Login to post a comment