AOPPA -- A program for extracting the AnarchyOnline resource database $Id: README 1112 2009-03-04 08:06:08Z os $ Copyright (C) 2002-2009 Oskari Saarenmaa . All rights reserved. Licensed under the GNU General Public License version 3. ============================================================================ Background, 'aoppa' is the current database parser I use to parse Funcom's CTree resource database and store it in a PostgreSQL database for use in my website (auno.org). This is the second, or perhaps third, generation of the parser, the first one was written entirely in PHP and had tons of problems. Including the fact that it took several hours to parse the database, in comparison it takes only about 5 seconds for aoppa to parse the current shadowlands database. As you can probably tell, this is mostly Unix stuff -- I don't do any development on Windows. The parser can only read a special data format that is produced by a Windows program that actually dumps the Anarchy Online CTree database. The program is called 'adbd' and the source code for it is available at http://auno.org/dev/adbd/ . I produced the program on a Linux system with the MinGW32 cross-compiler and run it under Wine. It would probably be trivial for someone familiar with Windows to rewrite such a program, the only thing it has to do is fetch records from the CTree database and then store them in a file in the format AOID: LEN: \n==========\n\n==========\n Plugins, Aoppa supports plugins, in fact it can't do anything at all without at least one plugin loaded. Currently there exist two kinds of plugins, 'input' and 'output' ones. Input plugins are used to feed data to the parser, and output plugins are used to store the data structures populated by the parser. Aoppa ships with four default plugins, input-mmap and input-gzip for reading data from files generated by ADBD (see above), and output-xml and output-ign for writing XML and IGN-bot format output files. There's also the aunoorg-output plugin that I will discuss a bit later. The plugin system is quite simple to use if you wish to create new plugins, you should just add a public structure of the type aoppa_plugin_t called 'aoppa_plugin' in your code and then compile it as a shared object. The aoppa_plugin structure contains the following elements (in order): api version - for specifying the aoppa API version plugin type - can be AOPPA_INPUT, AOPPA_OUTPUT or both init function - function to be called when plugin is loaded finish function - function to be called when plugin is unloaded store function - function to store item_t structures flush function - function to be called when done with input file parse function - function to be called with a file to parse description - a const char * with a short description of the plugin All of these, except the version, are optional and can be NULL if the specified functionality is not required or wanted in the plugin. See aoppa.h or the supplied plugins for more information. The obvious plugin that people would probably want, and the one I would like to see developed is one for reading data directly from the AO CTree database. That would of course require someone to port aoppa to Windows. Auno.org, There are several files in the repository that are directly related to my Auno.org website. These include a special PostgreSQL output plugin, the sql schemas and some tools for creating additional tables out of the data in aodb_* tables. Some of them might be interesting for others who want to develop an AO database website, most are probably uninteresting, though. Getting started, Get the latest .tar.gz package from http://auno.org/dev/aoppa/ and gunzip and untar it in your favourite Unix system. Then run 'make' to generate aoppa and the plugins and run: ./aoppa -p input-gzip.so -p output-xml.so Where the are your database dumps generated by ADBD. If you don't want or are unable to run ADBD you can also fetch 18.0.0 item and nano dumps from http://auno.org/dev/aoppa/items-18000000ep1.dat.gz and http://auno.org/dev/aoppa/nanos-18000000ep1.dat.gz After running the command you can just watch the XML output scroll by... Development, I have been meaning to release this code for public for as long as I have been working on it, but there's always been one or another thing that has blocked the release. I have wanted to fix some of the most obvious and embarrassing bugs in the code before a release, but the past has proved that the chances are that it'll never happen unless I just go ahead and give out the current version of the code. As you can see, I have chosen the GNU General Public License for the project. What it means is that you can freely copy, modify, sell, and distribute aoppa, but you have to give the full source code with any binaries you ship. This includes the source code for any derivative works, or anything that is linked with Aoppa. This includes all plugins. I would also like anyone who modifies the code to send diffs of their modifications back to me, so I can update my version and post any improvements on my website. If there is a lot of interest in this we can probably also setup a Sourceforge project. Anything and everything else, If you have any questions about this feel free to send me an email at auno@auno.org or write in the forums at http://auno.org/ Cheers, Auno