General Purpose Graphics Processing Unit (GPGPU)

Both ATI and nVidia have started opening their graphics processing units (GPUs) up to general purpose computation. The pipeline of pixel and vertex shaders have lots of transistors designed for 32-bit float matrix multiplication. In graphics, it's easy to parallelize and multiple computing units were etched into silicon to make the GPU faster.

ATI's effort, Close-to-Metal, is used by Folding@Home to be able to accelerate certain computations. I wanted to take a closer look, but their SourceForge project is temporarily shutdown.

Compute Unified Device Architecture (CUDA) is nVidia's project. I took a quick look at their documentation and didn't focus too heavily on the implementation details.

In assorted online discussions, the key limitation to the GPGPU processing is the bandwidth between main memory and the GPU. Without the ability to execute many branches or adequate integer processing, the data and code that gets stuck in GPU memory can't easily get out.

As a programmer, I'll optimize the data flow between the CPU and GPU for only things that I think might make payoff. I'll be spending the time redesigning those parts of the program that benefit from the GPU's speed at processing 32-bit floats.

Wouldn't it just be better if some lower-level software could help? Maybe the compiler or virtual machine, like Java or VMWare, could help without forcing me to redesign my program.