开发者

How to remove the last 2 lines of a huge file without reading it

开发者 https://www.devze.com 2023-04-08 19:35 出处:网络
I have a 100GB file, and need to remove the last开发者_运维技巧 two lines of it. I do not want to read from it since it will take about an hour to get to the bottom of it so sed does not seem to be an

I have a 100GB file, and need to remove the last开发者_运维技巧 two lines of it. I do not want to read from it since it will take about an hour to get to the bottom of it so sed does not seem to be an option.

My disk is also too small to be able to copy that file.

What are my options here ?

Thanks.


I'm pretty sure in .Net you can open a FileStream to a file, then move the pointer to a specific byte (which you could calculate), and modify it from there. However, I'm not sure if it has to pass the whole stream when you try to save it, so this might be more useful if you needed to copy just the last 2 lines.


In C/POSIX, you should be able to use fseek with the SEEK_END option to back up a little bit from the end of the file (say 512 bytes), then read those 512 bytes into memory.

From that, you can figure out exactly where the second last line starts and then use truncate or ftruncate to actually truncate the file at that point.

If the last two lines are more than 512 bytes (ie, the start of the second last line doesn't show up in the chunk you read), just increase the value to 1024 and try again. Keep going until you find it.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号