Bugzilla – Bug 53
High memory usage when using high table id
Last modified: 2010-12-15 12:33:03 PST
You need to
before you can comment on or make changes to this bug.
When replicating a database, the highest table id used in the configuration has
a direct impact on the memory usage of the slon processes for receiver nodes.
For example replicating a table using tableid = 1 might result in slon using 1
MB. but choosing a date-like id such as 20080623 causes the (resident) memory
usage to be about 160 MB.
remote_worker.c has the following comment about this:
* Remember the fully qualified table name on the fly. This
* might have to become a hashtable someday.
Yes, indeed, if you define a table with id 20080623, this establishes the size
of the array associated with wd->tab_fqname as being at least 20080623.
For now, I'll add documentation indicating this bottleneck.
I've moved this one to 2.0, since it isn't critical but should be fixed.
Instead of a hash table, this could also be implemented as a btree without
causing any dependencies, which would be a requirement to consider it for
Some further discussion has taken place on this; see
Possible BSD-licensed hash table implementations:
* [http://www.cl.cam.ac.uk/~cwc22/hashtable/ C Hash Table]
* See also [https://github.com/ryantenney/chashtable ryantenney / chashtable @
Used in Xen]
* [http://uthash.sourceforge.net/ UTHash]
* [http://burtleburtle.net/bob/c/lookup3.c lookup3] - Public Domain