Talk About Network

Google


Register and Login
Nick
Password
Register create new account Sign up is FREE and you can post replies, new topics, bookmark posts and more!
Recover lost password


Programming > Perl Beginners > Re: opening a b...
Latest [ Topics | Posts ] Archive Post A New Topic Post a Reply
<< Topic < Post Post 5 of 11 Topic 10959 of 11509
Post > Topic >>

Re: opening a big file

by Jenda@[EMAIL PROTECTED] (Jenda Krynicky) Apr 21, 2008 at 12:12 PM

From: Richard Lee <rich.japh@[EMAIL PROTECTED]
>
> Mr. Shawn H. Corey wrote:
> > It still has to go through the entire file and mark the offsets to the
> > start of every line.
> >
> > The best way to do this is just to bite the bullet and do it.
> 
> There is no way to read say last 10 MB of the file or something? It's 
> very surprising why no such thing exists..

It's a very different thing to read the last 10MB and to read the 
last 100000 lines. You know how many bytes are 10MB so you can seek() 
just to that position easily, you don't know (unless the lines are of 
the same and known length) how many bytes are 100000 lines. So if 
you'd really need to process the last 100000 lines in order, you'd 
first have to find the 100000th newline character counting from the 
end ... that is seek() say 1MB before the end, read 1MB, count the 
newlines, seek additional 1MB, read 1MB, count the newlines and 
repeat this till you find the chunk containing the 100000th newline, 
then find its position, seek() to that position (keep in mind that 
you have to read the file as binary, if you let Perl convert newlines 
or something like that, then the positions will not match!) and start 
reading.

Any chance you instead want to start reading where you left off last 
time? In that case you want to remember the position in the file 
using tell() whenever you finish processing it, store it somewhere 
and next time start by seek()ing to that position.

Jenda
===== Jenda@[EMAIL PROTECTED]
 === http://Jenda.Krynicky.cz
=====
When it comes to wine, women and song, wizards are allowed 
to get drunk and croon as much as they like.
	-- Terry Pratchett in Sourcery
 




 11 Posts in Topic:
Re: opening a big file
chas.owens@[EMAIL PROTECT  2008-04-20 16:07:27 
Re: opening a big file
rich.japh@[EMAIL PROTECTE  2008-04-20 17:02:58 
Re: opening a big file
rich.japh@[EMAIL PROTECTE  2008-04-20 18:10:02 
Re: opening a big file
krahnj@[EMAIL PROTECTED]   2008-04-20 15:28:18 
Re: opening a big file
Jenda@[EMAIL PROTECTED]   2008-04-21 12:12:37 
Re: opening a big file
david@[EMAIL PROTECTED]   2008-04-20 17:12:15 
Re: opening a big file
noreply@[EMAIL PROTECTED]  2008-04-21 17:18:02 
Re: opening a big file
rich.japh@[EMAIL PROTECTE  2008-04-21 11:19:10 
Re: opening a big file
noreply@[EMAIL PROTECTED]  2008-04-21 17:41:29 
Re: opening a big file
rich.japh@[EMAIL PROTECTE  2008-04-21 11:47:00 
Re: opening a big file
noreply@[EMAIL PROTECTED]  2008-04-21 18:25:26 

Post A Reply:
  Go here to Signup

AddThis Feed Button


About - Advertising - Contact - Frequently Asked Questions - Privacy Policy - Terms of Use - Signup

Contact
tan12V112 Wed Jul 23 20:58:13 CDT 2008.