Talk About Network

Google


Register and Login
Nick
Password
Register create new account Sign up is FREE and you can post replies, new topics, bookmark posts and more!
Recover lost password


Programming > Idl-pvware > Numbers from no...
Latest [ Topics | Posts ] Archive Post A New Topic Post a Reply
<< Topic < Post Post 1 of 9 Topic 5439 of 6249
Post > Topic >>

Numbers from nowhere?

by Conor <cmancone@[EMAIL PROTECTED] > Feb 15, 2008 at 07:08 AM

I'm just curious:

print,3.3,format='(f30.28)'
3.2999999523162841796875000000

Obviously float is only good to ~6 decimal places, but IDL will still
print out additional digits.  It seems that it does reach a point
where IDL will only print out zeroes.  I'm just curious - where do
these extra digits come from?  Does it come from rounding errors
converting binary to decimal?  Or does it come from some other
nefarious location?
 




 9 Posts in Topic:
Numbers from nowhere?
Conor <cmancone@[EMAIL  2008-02-15 07:08:24 
Re: Numbers from nowhere?
bennette@[EMAIL PROTECTED  2008-02-15 13:12:02 
Re: Numbers from nowhere?
Paul van Delst <Paul.v  2008-02-15 16:31:30 
Re: Numbers from nowhere?
David Fanning <news@[E  2008-02-15 15:51:16 
Re: Numbers from nowhere?
elwood <epolomsk@[EMAI  2008-02-17 12:19:44 
Re: Numbers from nowhere?
David Fanning <news@[E  2008-02-17 14:53:56 
Re: Numbers from nowhere?
Sven Utcke <utcke+news  2008-02-21 14:06:48 
Re: Numbers from nowhere?
Norbert Hahn <ihatespa  2008-02-21 17:29:03 
Re: Numbers from nowhere?
elwood <epolomsk@[EMAI  2008-02-25 09:57:14 

Post A Reply:
  Go here to Signup

AddThis Feed Button


About - Advertising - Contact - Frequently Asked Questions - Privacy Policy - Terms of Use - Signup

Contact
tan12V112 Fri Oct 10 14:46:47 CDT 2008.