Finding file names (from a directory) in a file

I wa开发者_JAVA百科nt to find if all the files of a certain type have been logged by my program. So essentially, I have a logfile with just the file names and then I use a function to run through the file to check if the file is present. Now the contents are huge but I did this in a crude way. Unfortunately, it's not working properly.

import subprocess
import sys
import signal
import shutil
import os, fnmatch


#open file to read
f=open("logs", "r") #files are stored in this directory
o=open("all_output_logs","w")
e=open("missing_logs",'w')


def locate(pattern, root=os.curdir):
    '''Locate all files matching supplied filename pattern in and below
    supplied root directory.'''
        #ignore directories- ignore works, just uncomment. 
    #ignored = ["0201", "0306"]
    for path, dirs, files in os.walk(os.path.abspath(root)):
        #for dir in ignored:
           # if dir in dirs: 
                #dirs.remove(dir)
        for filename in fnmatch.filter(files, pattern):
            yield os.path.join(path, filename)



    #here i log all the files in the output file to search in
for line in f:
    if line.startswith("D:"):
        filename = line
        #print line
        o.write(filename)

f.close()
o.close()
r.close()

i=open("all_output_logs","r")
#primitive search.. going through each file in the directory to see if its there in  the log file
for filename in locate("*.dll"):
    for line in i:
        if filename in i:
            count=count+1
            print count
        else:
            e.write(filename)

I do not see my dummy variable count being printed, and I only get one filename which is somehow in the middle of the list.

---

The problem is tha the lines in the file are read only in the first pass, and that a file object (i in your case) doesn't support the use of the in operator as you expect. You can change the code to something like:

lines = open("all_output_logs","r").readlines()
for filename in locate("*.dll"):
    for line in lines:
        if filename in line:
            count=count+1
            print count
        else:
            e.write(filename)

But it still would be inefficient and somewhat awkward.

Since you say that the log file is "huge" then you probably don't want to read it all into memory, so you'll have to rewind for each lookup:

f = open("all_output_logs","r")
for filename in locate("*.dll"):
    f.seek(0)
    for line in f:
        if filename in line:
            count=count+1
            print count
        else:
            e.write(filename)

I left the in operator because you don't specify what each line of the log file contains. One would have expected filename == line.strip() to be the correct comparison.