Quantcast
Channel: Sanity Phailed.me » floating point
Browsing latest articles
Browse All 4 View Live

Image may be NSFW.
Clik here to view.

Floating point quirks

Table Of Content 1. A closed form equation for 2. Uh oh 3. An explanation 1. A closed form equation for function S = S(n) S = 1 - 1/(1+n); end It’s a really simple problem, and if you do a little bit...

View Article


Image may be NSFW.
Clik here to view.

Fixing floating point cancellation I

This sounds like an easy problem, but it turns out that the solution isn’t so simple. If we plot , you’ll see that its graph is fairly smooth early on, but once approaches , it is no longer smooth. 1....

View Article

Image may be NSFW.
Clik here to view.

Introduction to Scientific Computing: Error Propagation

Table Of Content 1. Safety First 2. A Modern Day Little Gauss Story At a Brief Glance 3. Some Basics – Errors Error Propagation Arbitrary Differentiable Function Table of Error Propagation 4. Example...

View Article

Image may be NSFW.
Clik here to view.

0x5f400000: Understanding Fast Inverse Sqrt the Easy(ish) Way!

Table Of Content TL;DR – Start Here! 1. Introduction WTF? Obsession 2. Switching Strategy Converting between a Float and its Bits in memory Looking at b2f(f2b(x)*b) 3. Homing In on 0x5F400000 Adding...

View Article
Browsing latest articles
Browse All 4 View Live