Spurious ImportErrors (with modules importing submodules?)

I have a problem that I don't really know where to begin solving. Maybe it'll ring someone's bell.

TLDR: a Django app crashes and upon restart it runs but fails to import some modules. Upon another restart, all is fine again.

The wh开发者_StackOverflow社区ole story:

Different applications (we're up to three right now) on different Python (2.5.x, 2.6.x and 2.6.x) and Django versions (1.1.0, 1.2.5 and 1.3.0, respectively) occasionally exhibit spurious ImportErrors. For example, one of these apps started to fail every request by throwing an ImportError inside:

from django.contrib.gis.maps.google import GMarker, GEvent

We collected strace output and the relevant chunk is below (absolute path replaced with DIR for brevity and protecting the guilty).

stat64("DIR/django/contrib/gis/maps/google/GMarker", 0xf699ce3c) = -1 ENOENT (No such file or directory)
open("DIR/django/contrib/gis/maps/google/GMarker.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("DIR/django/contrib/gis/maps/google/GMarkermodule.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("DIR/django/contrib/gis/maps/google/GMarker.py", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("DIR/django/contrib/gis/maps/google/GMarker.pyc", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)

(and once again the same thing s/GMarker/GEvent/)

After restarting the process everything runs smoothly, also running:

python -c 'from django.contrib.gis.maps.google import GMarker'

produces no errors.

The GMarker and GEvent classes are actually defined in django.contrib.gis.maps.google.overlays and imported in ...maps/google/__init__.py:

from django.contrib.gis.maps.google.gmap import GoogleMap, GoogleMapSet
from django.contrib.gis.maps.google.overlays import GEvent, GIcon, GMarker, GPolygon, GPolyline
from django.contrib.gis.maps.google.zoom import GoogleZoom

so it's fully expected that loading GMarker.py et al. will fail. It seems that Python has somehow forgotten about __init__.py and its namespace.

The apps are relatively high-traffic and it is conceivable (though not certain) that they might have exceeded their VM limits and recovered almost-gracefully. Also, at least in two cases the app had earlier problems leading to a crash -- SIGSEGV in one case, and err... something else in the other). A single app restart caused it to throw ImportErrors around, and another one made it behave again. Corrupted .py[c]? Timestamps are ancient.

All these apps are running on flup the wsgi-to-fastcgi server.

Each of these apps failed once so far (in completely different modules, two cases of __init__.py "forgotten" but I cannot find the third error ATM) so I cannot tell whether the modules are somehow meaningful.

Any and all pointers and ideas appreciated!

---

Actually your strace line does not help a bit; those accesses did not result in the module being imported.

There might be multiple reasons for such import errors:

1. someone modifies sys.path and that is not within the module path anymore
2. a cyclical import in those modules, that is triggered by you importing from another module first!
3. you have site conflict (yep, had that kind of), same module being loaded from different site packages locations
4. you have a module in the search path conflicting with an other module somewhere, or with a system module.

If you would paste the SECOND set of strace lines, we would be closer to solution?

Update

For 2. I mean that if you have 2 files with the following structure

foo.py/init.py:

from bar import baz

bar.py/init.py:

import foo
def baz():
     pass

snafu.py:

import bar
import foo

ok.py:

import foo
import snafu

Run python snafu.py and you get a crash and similar strace output, run python ok.py and everything works.

---

The strace output seems suspicious to me:

DIR/django/contrib/gis/...

I am wondering about this DIR part. Is it possible you misstyped the PYTHONPATH variable somewhere, using DIRinstead of $DIR?

---

I'd suggest you to wrap the import with a try... except ImportError and add a code doing something like that (anything that logs, instead of print, will do)

import sys
print sys.modules["django.contrib.gis.maps.google"]
print dir(sys.modules["django.contrib.gis.maps.google"])

This should let you understand what's happening.