Strange data ending up in users' XML saved file_问答_开发者

My app uses XML to save user data to a file. I have just recently received 2 reports from users who are seeing completely unexpected data in their file. Instead of XML, it looks like this:

({"windows":[{"tabs":[{"entries":[{"url":"https://mail.google.com/a/cast...

And a bit more from the middle of the file, which weighs in at almost 30KB:

{\"n\":\"bc\",\"a\":[null]},{\"n\":\"p\",\"a\":[\"ghead\",\"\",0]},{\"n\":\"ph\",\"a\":[{\"gb_1\":\"http://www.google.com/

Can anyone tell me what kind of data this is, or how it's ending up in my users' data file? Both users reported having to hold down the power button to shutdown their machines. The shutdown in one case was a Firefox freeze, and in the other case was a mouse issue. One of the users actually experienced a kernel panic.

I'm not yet believing this is a memory-management issue, as my user base is more than 100,000 persons, and I've only received 2 reports. I'm thinking it's something more narrow/rare.

This is the piece of code I'm using to write data to my file:

NSString *xmlString = [[self convertContextToXmlString:context] retain];

NSError *e = nil;

[[xmlStr开发者_StackOverflowing dataUsingEncoding:NSUTF8StringEncoding] writeToFile:location options:NSDataWritingAtomic error:&e];
[xmlString release];
if (e) {
    NSLog(@"An error occurred saving: %@", [e description]);
}
return e;

Data saving never occurs on a background thread, always on the UI thread. Also I'm using the NSDataWritingAtomic option to write the data to a file.

Edit: The second user's file has nearly identical data. So both erroneous contents are coming from the same place, but where? I'll be adding a 200 point bounty to this question as soon as I'm able.

AV/////wEAAAAAAAAAAAABAAA="}]}]},{"url":"http://googleads.g.doubleclick.net/pagead/ads?client=ca-pu

Edit 2: Received a third report from a user who also experienced data corruption after holding down the power button to shut down his machine. His data had a lot of random junk at the beginning, and then proper data at the end:

(garbage)rred="1"><rest of it was normal xml...>

Received an excellent answer from one of the Apple devs. Will be porting my existing model to Core Data over the next few weeks. (StackOverflow messed with some of the lists/formatting, but it's for the most part still very readable.)

I'll start my response with a word about HFS Plus journalling. Since the introduction of journalling in Mac OS X 10.2.x, Mac OS X's file system correctness guarantee has been that--regardless of kernel panics, power failures, and so on--file system operations will result in one of two outcomes:

o either the operation will be rolled forward by the journal, in which case it will be as if the operation had completed successfully

o or the operation will be rolled back, in which case it will be as if the operation never happened

This guarantee has two critical limitations:

o It applies to individual file system operations (creates, deletes, moves, and so on), not to groups of operations.

o It applies to the logical file system structure only, not to the data within files.

In short, the purpose of the journal is to prevent overall file system corruption, not the corruption of a specific file.

Keeping this in mind, let's look at the behaviour of -[NSData writeToFile:options:error:]. Its behaviour can be very complex but is quite simple in the typical case. One way to explore this is to write some code and look at its file system behaviour using . For example, here's some test code:

- (IBAction)testAction:(id)sender
{

BOOL success;
NSData * d;
struct stat sb;

d = [@"Hello Cruel World!" dataUsingEncoding:NSUTF8StringEncoding];
assert(d != nil);

(void) stat("/foo", &sb);
success = [d writeToFile:@"/tmp/WriteTest.txt" options:NSDataWritingAtomic error:NULL];
(void) stat("/foo", &sb);
assert(success);
}

The two calls to are just markers; they make it easy to see which file system operations are generated by -writeToFile:options:error:.

You can look at the file system behaviour using:

$ sudo fs_usage -f filesys -w WriteTest

where "WriteTest" is the name of my test program.

Here's an extract from the resulting fs_usage output:

14:33:10.317 stat [ 2] /foo 14:33:10.317 lstat64 private/tmp/WriteTest.txt 14:33:10.317 open F=5 (RWC__E) private/tmp/.dat2f56.000 14:33:10.317 write F=5 B=0x12 14:33:10.317 fsync F=5 14:33:10.317 close F=5 14:33:10.318 rename private/tmp/.dat2f56.000 14:33:10.318 chmod private/tmp/WriteTest.txt 14:33:10.318 stat [ 2] /foo

You can clearly see the "stat" calls that surround the -writeToFile:options:error: call, meaning that all the stuff between those calls is generated by -writeToFile:options:error:.

What does it do? Well, it's actually pretty simple:

It creates, writes to, fsyncs and closes a temporary file containing the data.
It renames the temporary file on top of the file you're writing to.
It resets the permissions of the final file.

All-in-all this is a pretty standard UNIX-style safe save. But the question is, how does this impact on data integrity? The key thing to note is that fsync does not guarantee to push all data to the the disk before returning. This issue has a long and complex history, but the summary is that fsync is called way too many times, in way too many performance-sensitive locations, for it to make that guarantee. This means that all of the file corruption problems you're seeing are possible as explained below:

o "iProcrastinate_Bad_2.ipr" and "iProcrastinate_Bad_3.ipr" just contain the wrong data. This can happen as follows:

App creates temporary file.
App writes to temporary file. In response to this the kernel:

a. allocates a set of blocks on disk b. adds them to the file c. extends the file's length d. copies the data written to the buffer cache

App fsyncs and closes the file. The kernel responds by scheduling the blocks of data to be written ASAP.
App renames the temporary file on top of the real file.
System kernel panics.
When the system reboots, the changes from steps 1, 2a..2c, 3 and 4 are recovered from the journal, meaning that you have a valid file that contains invalid data.

o "iProcrastinate_Bad_1.ipr" is just a slight variation of the above. If you open the file with a hex editor, you'll find that it looks good except for the range of data at offset 0x6000..0x61ff, which seems to contain data totally unrelated to your app. Notable the length of this data, 0x200 bytes, is exactly one disk block. So it seems that the kernel managed to write all of the user data to disk except for this one block.

So where does this leave you? It's unlikely that -[NSData writeToFile:options:error:] will ever get more robust than it already is; as I mentioned earlier, changes like this tend to have a negative effect on overall system performance. This means that your app will have to take care of this problem.

There are three common ways to harden your app in this regards:

A. F_FULLFSYNC -- You can commit a file to permanent storage by calling with the F_FULLFSYNC selector. You could use this in your app by replacing -[NSData writeToFile:options:error:] with your own code that called F_FULLFSYNC instead of fsync.

The most obvious drawback to this approach is that F_FULLFSYNC is very slow.

B. journalling -- Another option to adopt a more robust file format, one that supports journalling perhaps. A good example of this is SQLite, which you can use directly or via Core Data.

C. safer save -- Finally, you could implement a safer save mechanism by way of a backup file. Before calling -[NSData writeToFile:options:error:] to write your file, you could rename the previous file to some other name, and leave that file around just in case. If, upon opening the main file you discover it's corrupt, you would then automatically revert to the backup.

Of these approaches my preference is for B, and specifically approach B with Core Data, because Core Data offers lots of benefits above-and-beyond data integrity. However, for a quick fix option C is probably your best bet.

Let me know if you have any questions about this stuff.

Logically I can see only two reasons for this to happen:

[self convertContextToXmlString:context] returned that string. In which case, we can't debug any further without some idea about how that method works. You could possible put in some kind of assertion to make sure the returned value looks like XML
Some other process/app/code is writing to the same location. Your app doesn't work with JSON at you say, so that would seem to rule out possibility it is you. What is this location?

That looks like JSON.

Where does the data come from? If it's a web service not under your control, did the vendor change the default response format and you aren't asking for XML explicitly?