Consolidation, RRD and some testing

I previously had a universal share called “share” shared on one machine and mounted by the rest. With MFS setup I have now removed share and copied the contents to MFS remounting it in the same place so that now it is decentralized and redundant.

Afterwards I had to rebuilt RRDtool with –disable-mmap as it was getting mmap related errors with MFS mounted file systems. This is most likely due to CentOS using old libraries.

A test crawl was run to make sure everything was still working…Depth: 5, 300k pages, 55 minutes.

I do need to replace CentOS with Ubuntu for all the latest pyRdfa code to work, a system image is currently being created.

The pyRdfa to Sesame process now correctly URLEncodes the baseURI and context, however I cannot test this in the crawler at the moment as pyRdfa runs very badly on Centos when using Lax mode (some html5 issues).

CentOS will be gone soon – it is too out of date for my needs.

Leave a Reply