Linux

Building and Installing libexodus

Tested on Centos 5.2/Boost 1.38/Postgres 8.3/Gnu Compiler 4.1.2

It might work with versions as far back as Boost 1.35 and Postgres 8.1 perhaps with minor tweaks.

libexodus Dependencies

Installing g++ compiler

yum list|grep gcc

yum install gcc-c++.i386

or for Ubuntu

sudo aptitude install build-essential

Installing Boost

http://shoddykid.blogspot.com/2008/07/getting-started-with-boost.html

Required for complete build of Boost - but those bits of Boost may not actually be required by Exodus.

yum install python-devel
yum install bzip2-devel

Required for unicode support in regex

yum install libicu-devel

or on Ubuntu

apt-get install libicu-dev

Download Boost Source

wget http://downloads.sourceforge.net/boost/boost_1_38_0.tar.bz2?use_mirror=garr

Unpack it

tar xvf boost_1_38_0.tar.bz2
cd boost_1_38_0

Become superuser to build and install it.

su

./configure --help
./configure --with-libraries=date_time,filesystem,iostreams,program_options,regex,system,thread

Optionally to speed the build of only the library versions required, ADD additional config to the BJAM_CONFIG line as follows:

nano Makefile

BJAM_CONFIG= xxxxxxxxxxxxx -j4 variant=release link=shared

Then build and install

make
make install

Skip to the next section if on Ubuntu, this part is unnecessary.

Add "/usr/local/lib" to the list of lib directories to ensure that the boost libs can be found.

nano /etc/ld.so.conf

Ensure the new config is operational.

ldconfig

Ensure boost includes can be found

ln -s /usr/local/include/boost-1_38/boost /usr/local/include/boost

Quit superuser

exit

Downloading and Building libexodus

libexodus.so is a shared library that allows you to write c++ programs with the style and semantics of pick basic. To support pick database style operations, it requires pgexodus to be built and installed too.

To give some insight into the automatic procedure see Manual Building with g++

Ensure you have postgres client development library (libpq-fe.h file)

yum install postgresql-devel

apt-get install libpq-dev

Download Exodus

cd ~
mkdir exodus
cd exodus
svn co http://svn.neosys.com/svn/trunk
cd trunk/exodus/exodus

Build and Install

./configure
make
make install

A couple of installation steps that have to be done by hand at the moment

ldconfig
ln -s /usr/local/include/exodus-9.6/exodus /usr/local/include/exodus 

Test that you can down develop in c++ with Exodus. The libs and includes will no longer be required with a better installer.

nano tester.cpp

#include <exodus/exodus.h>
int main() {

date().oconv("D").outputln();

}

g++ tester.cpp -lexodus-9.6 -lpq -I/user/local/include/exodus
./a.out
25 JUN 2009

Building and Installing pgexodus

pgexodus.so is a shared library addin c function for postgres to perform sort/select/index functions otherwise it is not required.

pgexodus Dependencies

Installing Postgres

Version 8.3

yum install postgresql

or

sudo aptitude install postgresql-8.3

Look for all available postgres packages and if they are installed

yum list|grep postgres

yum install postgresql-devel

or for Ubuntu

sudo aptitude install postgresql-dev
sudo aptitude install libpq-dev

Locate libpq-fe.h to check. Probably in /usr/include/postgresql/ or /usr/include/postgresql/8.3/server/ on Ubuntu

updatedb
locate libpq-fe.h

=== Building pgexodus

Currently hand built without even a make file. standard configure/make using autoconf/automake will be available soon http://www.lrde.epita.fr/~adl/dl/autotools.pdf

Note that all routines are marked for "C linkage" in order to work with the postgres library "pq". For convenience they are all compiled under C++ since callexodus requires boost which is c++. The .c routines could equally well be compiled with the plain c compiler. The link stage must be done in c++ style in order to link to the c++ runtime library "rt".

cd ~/exodus/trunk/exodus/pgexodus/src

g++ -fpic -c pgexodus.c extract.c callexodus.cpp naturalorder.cpp pgnaturalorder.cpp -I `pg_config --includedir-server` -I /usr/local/include/boost-1_38
g++ -shared -o pgexodus.so pgexodus.o extract.o callexodus.o naturalorder.o pgnaturalorder.o -lrt -lpq

Check the build as follows:

ld pgexodus.so

Should give something like the following. These 8 references to the postgres library all seem to be resolved at runtime.

pgexodus.so: undefined reference to `Float8GetDatum'
pgexodus.so: undefined reference to `pg_atoi'
pgexodus.so: undefined reference to `CurrentMemoryContext'
pgexodus.so: undefined reference to `pg_detoast_datum'
pgexodus.so: undefined reference to `pfree'
pgexodus.so: undefined reference to `elog_start'
pgexodus.so: undefined reference to `MemoryContextAlloc'
pgexodus.so: undefined reference to `elog_finish'

Also check the following

ldd -u pgexodus.so

Should give something like:

./pgexodus.so: error: symbol lookup error: undefined symbol: CurrentMemoryContext (continued)
undefined symbol: CurrentMemoryContext  (./pgexodus.so)
Unused direct dependencies:

        /lib/librt.so.1
        /usr/lib/libpq.so.5
        /lib/libm.so.6
        /lib/libgcc_s.so.1

Installing pgexodus

Copy the library to the postgres library folder

sudo cp pgexodus.so `pg_config --pkglibdir`

Register the c functions in postgres

sudo su - postgres
psql
\connect exodus

-- cut and paste the following SQL to register the functions into postgres --

CREATE OR REPLACE FUNCTION exodus_call(bytea, bytea, bytea, bytea, bytea, int4, int4) RETURNS bytea AS 'pgexodus', 'exodus_call' LANGUAGE C IMMUTABLE;

CREATE OR REPLACE FUNCTION exodus_extract_bytea(bytea, int4, int4, int4) RETURNS bytea AS 'pgexodus', 'exodus_extract_bytea' LANGUAGE C IMMUTABLE;

CREATE OR REPLACE FUNCTION exodus_extract_text(bytea, int4, int4, int4) RETURNS text AS 'pgexodus', 'exodus_extract_text' LANGUAGE C IMMUTABLE;

CREATE OR REPLACE FUNCTION exodus_extract_sort(bytea, int4, int4, int4) RETURNS text AS 'pgexodus', 'exodus_extract_sort' LANGUAGE C IMMUTABLE;

-- Remaining functions are STRICT therefore never get called with NULLS -- also return NULL if passed zero length strings

CREATE OR REPLACE FUNCTION exodus_extract_text2(bytea, int4, int4, int4) RETURNS text AS 'pgexodus', 'exodus_extract_text2' LANGUAGE C IMMUTABLE STRICT;

CREATE OR REPLACE FUNCTION exodus_extract_date(bytea, int4, int4, int4) RETURNS date AS 'pgexodus', 'exodus_extract_date' LANGUAGE C IMMUTABLE STRICT;

CREATE OR REPLACE FUNCTION exodus_extract_time(bytea, int4, int4, int4) RETURNS time AS 'pgexodus', 'exodus_extract_time' LANGUAGE C IMMUTABLE STRICT;

CREATE OR REPLACE FUNCTION exodus_extract_datetime(bytea, int4, int4, int4) RETURNS timestamp AS 'pgexodus', 'exodus_extract_datetime' LANGUAGE C IMMUTABLE STRICT;

Check the functions loaded properly

\df exodus*

Quit postgres command prompt and quite postgres user

\q
exit

For more information see http://www.postgresql.org/docs/8.1/static/app-psql.html

Setting up Postgres for Exodus

become user postgres and get into postgres command prompt

sudo su - postgres
psql

create a user and database for exodus

CREATE ROLE exodus LOGIN
 PASSWORD 'somesillysecret'
 CREATEDB CREATEROLE;

CREATE DATABASE exodus
 WITH ENCODING='UTF8'
   OWNER=exodus;

1. quit postgres command prompt and quite postgres user

\q
exit

Allow login from 127.0.0.1 using username/unencrypted password (instead of only logged-in username without password) http://developer.postgresql.org/pgdocs/postgres/auth-pg-hba-conf.html

nano /var/lib/pgsql/data/pg_hba.conf

or for Ubuntu

nano /etc/postgresql/8.3/main/pg_hba.conf

and add/edit these lines

#host    all         all         127.0.0.1/32          ident sameuser
host    all         all         127.0.0.1/32          password
hostssl all         all         0.0.0.0/0             password

service postgresql reload

or for Ubuntu

sudo /etc/init.d/postgresql restart

Building service

service is an incomplete multithreaded framework to support dynamic loading of library routines and serving a request/response queue.

www provides an optional web interface to the request/response queue. Windows only at the moment due to a single asp files but could be ported to php.

Building service

cd ~/exodus/trunk/service/service

g++ -fPIC -c *.cpp -I/usr/local/include/boost-1_38 -I../../exodus/mv/src
g++ -shared -o libservice.so -lc *.o
g++ -o service main.cpp -I/usr/local/include/boost-1_38 -L. -L/usr/local/lib -lexodus -lpq -lboost_filesystem-gcc41-mt\
-lboost_regex-gcc41-mt -lboost_thread-gcc41-mt -L../../exodus/mv/src -I../../exodus/mv/src -L./ -lservice

Running service

export LD_LIBRARY_PATH=../../exodus/mv/src:.:/usr/local/lib:$LD_LIBRARY_PATH

./service

Developing libexodus/pgexosdus/service

Committing your developments

First get a username and password by running the following unix command and send the output to an exodus subversion administrator eg steve.bush@neosys.com

htpasswd -nm YOURDESIREDUSERNAME

The administrator needs to run the following command as root and enter the encrypted pass (the bit AFTER the :) ignoring the warning about plain text password because it is already encrypted.

htpasswd -p /etc/apache2/dav_svn.passwd YOURDESIREDUSERNAME