Trace:

saturation_arithmetic

This shows you the differences between two versions of the page.

— |
saturation_arithmetic [2021/04/09 13:00] (current) richardrussell created |
||
---|---|---|---|

Line 1: | Line 1: | ||

+ | =====Saturation arithmetic===== | ||

+ | |||

+ | //by Richard Russell, 14th March 2021// | ||

+ | |||

+ | When working with audio or video data one often wants to perform 'saturation arithmetic', such that the result of an arithmetic operation is 'clamped' or 'clipped' to a certain range of values. So typically that might be signed audio samples, which you want to be limited to the range -32768 to +32767 (or -32767 to +32767, the difference is unlikely to be important) without 'wrapping around'. | ||

+ | |||

+ | Obviously that is easily achieved using conditional tests (IF statements) but commonly the data will be in an array and it would be very useful if array arithmetic could be used. This means working out a way to perform the saturation operation without any conditional code. This is my attempt to do exactly that, the array a%() initially contains values in the range -65536 to +65535 and at the output contains that data saturated to the range -32767 to +32767: | ||

+ | |||

+ | <code bb4w> | ||

+ | t%() = a%() DIV &8000 | ||

+ | t%() OR= t%() DIV 2 | ||

+ | t%() *= &7FFF | ||

+ | a%() AND= t%() - 1 | ||

+ | a%() OR= t%() | ||

+ | </code> | ||

saturation_arithmetic.txt ยท Last modified: 2021/04/09 13:00 by richardrussell

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International