|Summary:||High memory usage when using high table id|
|Product:||Slony-I||Reporter:||Damian Quiroga <dquiroga>|
|Component:||slon||Assignee:||Slony Bugs List <slony1-bugs>|
Description Damian Quiroga 2008-06-23 07:32:09 UTC
When replicating a database, the highest table id used in the configuration has a direct impact on the memory usage of the slon processes for receiver nodes. For example replicating a table using tableid = 1 might result in slon using 1 MB. but choosing a date-like id such as 20080623 causes the (resident) memory usage to be about 160 MB.
Comment 1 Christopher Browne 2008-06-23 08:57:40 UTC
remote_worker.c has the following comment about this: /* * Remember the fully qualified table name on the fly. This * might have to become a hashtable someday. */ Yes, indeed, if you define a table with id 20080623, this establishes the size of the array associated with wd->tab_fqname as being at least 20080623. For now, I'll add documentation indicating this bottleneck.
Comment 2 Jan Wieck 2010-08-13 19:28:54 UTC
I've moved this one to 2.0, since it isn't critical but should be fixed. Instead of a hash table, this could also be implemented as a btree without causing any dependencies, which would be a requirement to consider it for 2.0.x.
Comment 3 Christopher Browne 2010-12-03 13:14:45 UTC
Some further discussion has taken place on this; see [http://wiki.postgresql.org/wiki/SlonyBrainstorming#.2353_-_high_RAM_usage_with_high_table_.23] Possible BSD-licensed hash table implementations: * [http://www.cl.cam.ac.uk/~cwc22/hashtable/ C Hash Table] * See also [https://github.com/ryantenney/chashtable ryantenney / chashtable @ GitHub] * Note: [http://xen.1045712.n5.nabble.com/Legal-concerns-added-hashtable-implementation-td2485274.html Used in Xen] * [http://uthash.sourceforge.net/ UTHash] * [http://burtleburtle.net/bob/c/lookup3.c lookup3] - Public Domain