running a python script indefinitely (as a process, pretty much)

i have tests that i ran which can take up to 15m at a time. during these 15m, a log file is periodically written to. however, most of the content is useless.

in response to this i have a python script that parses out the useless text and displays the relevant data.

what i'm trying to achieve is similar to what tail -f log_file, constantly updating the terminal with the newest additions to a file. i was thinking that if a python script ran as a process, it could parse the log file whenever the tests write to it, then the python script can go to sleep until interrupted again once the log file is written to.

any ideas how one can achieve this?开发者_开发知识库

i already have a script that does the parsing, i just don't know how to make it do it continually and efficiently.

---

You could just have the script filter standard input, and pipe tail -f through it. When you're waiting on stdin, your script will sleep, so it's plenty efficient.

Eg.

python long_running_script.py && tail -f log_file | python filter_logs.py

Your script can be something like

while true:
    line = sys.stdin.readline()
    if filter_line(line): print line

---

looks like you need something like "pytailer": http://code.google.com/p/pytailer/

While I never used it myself, last example looks like what you want.

---

any ideas how one can achieve this?

This should be pretty easy to do. Most of what you want is already part of your OS.

python test.py | python log_parser.py

Be sure your tests write their log to stdout instead of some other file. This is often easy to do with small changes to the logging configuration.

---

Having implemented almost this exact tool, I had great success using the inotify capability in twisted