crschmidt: Primary reason being that Bradfitz (current owner) loves tech, doesn't love business.
crschmidt: One thing that LiveJournal will do is (finally cough) implement trackback
crschmidt: One thing that LiveJournal will do is (finally cough) implement trackback
PhUrl: For reference, input file
PhUrl: You can see that there is a great overhead of using redland perl interface to process n3
Phurl: Here is a small c program to read triples from a binary file and determine frequency of the in arcs, parse fast.c
Phurl: updated to be able to read and process the input file 1000 times in 20 seconds, mean of times 52 times per second!
Phurl: The c++ compiler version needs 2 seconds longer
Phurl: Test ran on a AMD Athlon(tm) XP 1600+, 1394.086 mhz, 1gb ram, 128k L1 cache, 256 L2 cache
Phurl: The New format,N3V(Binary N3 Vector file), is here Input File
Phurl: This format only handles resources atm, so, statements like uri,uri,uri
Phurl: Encoding each uri into an integer index, it is completly context specific and needs a mapping file of index<-->uris to be decoded
Phurl: Forgot to mention that this program is derived from the source of the ultra fast gnu grep tool, and it is under the GPL license :)
Phurl: Compile with gcc-3.3 -lm -O3 parsefast.c
Phurl: n3v converter use like file:../data/dump.ntriples map.n3 pnet_cscc_C_semantics_introspector-dump.n3v output.debug SCS
Phurl: The converter can use any encoding that perl pack allows, Here is an overview, I have used subject=unsigned short,pred=unsigned char,object=unsigned short is encoded like SCS. LLL would be three longs.
Phurl: Benchmark script benchmark script executes Shell Script 1000 times and times it.
Phurl: by shortening the triples to 6 bytes, I get a 1/3 performance increase, due to more optimal cache usage.
Phurl: Wrote up a bit about this here