Use relative pointers within shared-memory tables

[bovine]

Currently shared-memory tables store physical pointer addresses of other records and strings within shared-memory. This has the side effect of necessitating enforcement that attaching to an existing shared-memory table is able to be allocated at the same memory address. In general, it is not possible for the OS to guarantee that the same virtual address location will be available for mmap() in another process so the current method is really only well-suited for use by processes that inherit the table through forking, or independent processes that happen to be lucky when calling mmap.

Switching all absolute pointers to be relative pointers (with respect to the address of the shared-memory block) will have a slight performance penalty but will allow greater flexibility. By using a preprocessor #define around all pointer operations, it would be possible to conditionally switch on/off use of relative pointers at compile-time if we believe the performance hit is significant.

[resuna]

The use of absolute pointers allowed shared memory speedtables to be implemented without modifying all the speedtables code to support shared memory. Most of the code doesn't treat shared and private speedtables any differently. For one example, the only modification in the skiplist code was to pass the shared memory pool to the allocator, and change the order of some of the operations on skiplists to guarantee that the structures were always internally consistent without locking. I'm not even sure that guarantee will hold with relative pointers, since pointer indirection wouldn't be atomic any more.

[lehenbauer]

Agreed, this would be very difficult to achieve and would affect non-shared-memory speedtables performance negatively, about for sure. The atomicity issue Peter brought up could be solved with locks, but that could in a ginormous amount of locking.

I think with a 64-bit address space it wouldn't be difficult to map multiple shared memory speedtables to specific addresses in a non-conflicting way at the cost of a slight hassle. This will require multiple shared memory speedtables to actually work, which it doesn't currently, due to some static variables in ctables/shared/shared.c.

[bovine]

Boost supports relative pointers in shared-memory regions using the offset_ptr template:
http://www.boost.org/doc/libs/1_51_0/doc/html/interprocess/offset_ptr.html

Although it still incurs a performance penalty, it might be worth investigating. The 64-bit address space does potentially mitigate the need for this, as long as an unused region of memory can be initially located.