Difference between memcache and python dictionary

In my current project, I am using Memcache to store key-value pairs, but since the communication happens over the socket between my process and the Memcache causing the huge latencies. We went with memcache because we had a requirement of storing large amount of key-value pairs. But now I want to store the dictionary as a global datastructure in my process. Is it a good thing? Because the dictionary will be stored in proc开发者_JAVA技巧esses address space. Suggestions please....

---

The usual reason to use memcached is that you would like to distribute the cache among multiple machines, with the goal of both having data available on all the machines, while also utilizing the storage of all the machines. If those requirements don't apply to you, and you only need the cached data on a single machine, then memcached doesn't offer you all that much. In that case, moving the dictionary into your local process might be a good idea.

---

I wrote a thorough answer to this on the memcached "about" page. I drew pictures and everything.

In summary: If you have more than one process, the dictionary won't help you. If you have more than one process/computer, you're going to be burning tons of memory that could be reused in great ways that save you lots of money and get you more bigger stuff.

---

If you data is not so big, you may just dump your python dictionary to files with cPickle.dump or marshal.dump and, reload it from file with cPickle.load or marshal.load, and if you need to worry about diskspace, you may use bz2 or gzip compress / decompress during file read / rewrite.