Term Project – Final Report CSCE 4313 Programming Languages – Fall 2008 Importing and Exporting Data Into Open Simulator and Second Life John D. Hart Import-Export Abstract Second Life and, to a lesser extent, Open Simulator, have significant barriers to importing outside data. Prim size, file format, and barriers designed to prevent copying the creations of others all keep files out of Second Life. However, using a combination of parsing and converting techniques, it is possible to import outside data, though it is much easier in Open Simulator. Exporting is significantly easier, though data must then be converted into data usable by other programs. 1. Problem The problem our team faced was the problem of finding ways to import data for real world objects into Second Life and Open Simulator and to also export data from those programs into more recognizable file types. The problem was designed primarily around importing building and terrain data from various programs and converting it into compositions of prims that formed a building within Second Life and Open Simulator. My part in this problem was in converting data for buildings into data for objects suitable for prims. Prims have a maximum object size of 10m per side, and most buildings are larger than that. Thus, it was my job to divide each building into a series of smaller objects, none of which was greater than 10m on any one side. 2. Objective The objective of the xxx team was to be able to take terrain and building data, feed it into a series of programs, and then have those programs load the buildings and terrain into Second Life. The objective of my subprobject was to parse buildings down into objects that could be used as prims. Each building had to be constructed of ‘bricks’ which were no more than 10m on any one side. 3. Related Work One of our group members, Chad Ferrari, searched first for existing solutions. Using a prim ‘blender’ and the work of another group member, Matt Miller, he is able to convert prims to Collada. It cannot convert 3D models into a prim based format however. Along with the script for blender, there's also a SL script designed to parse the .prims file and make the objects in SL. I, however, have not been provided with links to this software. Term Project – Final Report CSCE 4313 Programming Languages – Fall 2008 Matt Miller created a GUI Import/Export tool. This allows you to log into a local opensim server by default. You can then search for objects. You will be given a list to choose from, give it a name, and export it to a .xml file. You can also select a .xml file with the proper format namely SLXML to import. That is on the Login Tab. On the Parse tab, you can select a .xml file to parse and it will give you the information or values of that xml/prim. Then there is the Convert tab. There are 3 converters. One converts from a .prims file or a .txt file with the same format, which is a prims.blender file to a .xml file which can then be imported. One converts from a slxml .xml file to a .prims file or .txt which can then be imported to prims.blender and converted to any other format blender supports including Collada. Then there is the Second Life Exported. This is a second life script he created. You put it on the Second Life, object you want to export. Touch the Object. It will then instant message you a series of lines that are in the same format of the .prims files. Open your chat box and copy the whole thing to a file, something.prims or .txt. This is where the 3rd converter comes in, it strips all the extra stuff such as time and object name that is included when you copy the text. This file can then be opened by prim.blender or converted to slxml .xml and imported with the tool into opensim. Casey Bailey completed KML/Collada importer capable of traversing a full COLLADA spec document embedded within Google Earth's KML. Returns model information including vector arrays and applied materials/textures. It also comes with a helper project that implements a GUI system for interacting with the class and viewing data. I wrote a program that would accept numerical data giving vertices and edges for objects and convert them into objects small enough to be used as prims. I was working on making it capable of taking in .kml files from Casey’s parser, but when I went to save the day before the due date, my computer crashed and my code became corrupted. I still know the basic pseudo-code and methods of making the program work, but the code itself was quite complex and I was unable to re-create it in a single night. 4. Architecture 4.1 Design The overall design of the project was: Search for existing import/export programs and concepts to incorporate into our project. Work on parsing data for massive buildings and terrain features into segments usable by Second Life Convert that data into data which was usable by Second Life Import that data into Second Life and ensure an adequate fit. Export data from Second Life to be modified in other programs, then re-imported. The design of my subproject was: Read in data for massive buildings Convert that data into simpler data that could be more easily worked with Term Project – Final Report CSCE 4313 Programming Languages – Fall 2008 Separate the data into distinct objects which could be easily manipulated Further divide those objects into simple rectangular objects Further divide those objects into objects which were less than 10m per side Write those objects out to a file in the same format as the original 4.2 Testing I was unable to finish and test the file conversion of my code before the crash. To test the basic algorithm, I used a quick and dirty GUI for entering vertices of the object. The program would then graph those vertices in a 3D interface and parse a new object once every 2 seconds, displaying the lines dividing parsed objects on the screen. Eventually, I added a quick and dirty file interface that accepted coordinates and then wrote over the file with the coordinates of the new objects. 5. Results The program I created for parsing the objects was capable of handling any object consisting entirely of right angles. The way the code worked was that it read in a single vertex as a starting point, then read in all connected vertices, four vertices total for an object that was only right angles. It then extrapolated the remaining four vertices that would make up a rectangular object. It would then record those as an object stored for export. It would then compare each point on the new object to the points on the old object. If there was a corresponding vertex (meaning it was on both the new object and the original) then that point was deleted from the original. This would remove the chunk that was parsed from the old object. Vertices that were not found on the old object were added to it and connected to others existing vertices, essentially closing it up and ensuring that the only the chunk that had been parsed was removed. The remaining object Term Project – Final Report CSCE 4313 Programming Languages – Fall 2008 was then tested to ensure there were vertices left and fed recursively back into the same algorithm. The second algorithm divided these rectangles into prim-sized objects. Each dimension (height, width, depth) was divided by 10 meters (round up) to determine the number of pieces the object would have to be subdivided into in that direction. That dimension was then divided by that number to see how big each object would be. Vertices would then be added and connected at each interval until it reached the end of the object. After running for all three dimensions, the original object, no matter how massive (excepting hardware limitations on variable size), was divided into rectangular objects that were close to, but not over, 10m on a side, allowing them to be imported into Second Life. 6. Conclusions 6.1 Summary First lesson learned: prepare for the worst and remember where your backup is. Humor and regret aside, this project has taught me much about the difficulty of conversion between multiple platforms. Through trying to convert massive amounts of data into something as simple as the objects we were learning, I learned not only the difficulty of converting from, say, .kml to .slxml or to actually figure out how to bridge the gap from my computer to the server, but also many interesting techniques for doing such. I learned how to write a program which would recognize common and ignorable lines in many common formats and also write relevant data back to the location it was extracted from, as well as formatting an export file of a different file type to accept the new data. I also learned how to alter data within a file while maintaining file integrity. 6.2 Impact Our work as a group may one day allow large companies and universities to import entire buildings, or even entire campuses, quickly and accurately into an environment such as Second Life, allowing those entities to give tours and even run test operations online, without the typical cost and logistical problems associated with doing that in the real world. They can also use this to make friendly user-interfaces for people wanting to contact or know more about the company or other client. My work individually will allow others to convert anything too massive for a certain system into smaller, more manageable objects which can then easily be imported. Large buildings can be dynamically converted into ‘bricks’ which can be dealt with individually to allow easier pipelining of the data. 6.3 Future Work Unless given an extension, a future student will have to re-create my source code given the methods above. Otherwise, a future student would have to add options onto the code for converting data of different file types and incorporate it into other parts of the project in order to allow the full work of importing building data in the second life to function. Term Project – Final Report CSCE 4313 Programming Languages – Fall 2008 Comments What did you learn? That I need to keep track of my backups in case something like this happens. I learned quite a few useful things about how to convert data from one format to another, including how to maintain object properties and how to turn one piece of data into many. What did you like about the class? I liked both the practical nature of the project and the fact that the information seemed to be useful and practical in the real world…of programming. What didn’t you like about the class? I was somewhat disheartened by the complexity of some of the test questions, as well as overwhelmed by the seemingly daunting project. I also didn’t like that it was at 8:00 am