Streams and Multi-GPU using CUVI

From CUVI Wiki
Revision as of 15:34, 4 May 2012 by Jawad (talk | contribs)

Using Streams with CUVI

CUVI framework provides a way to use streams with minimal coding effort. Each function call in CUVI takes an optional parameter to specify the stream on which it should run. The code below shows how a simple function call of CUVI can be divided into streaming calls on GPU. For most of the cases this will result in better performance as copying image data to GPU and processing that data on GPU happens simultaneously.

CUVI example

In this example we use CUVI's RGB2Gray function from Color Operations module on a full HD input image

//You may notice that all the functions below take an optional parameter for stream
//If the users doesn't wish to use it the program will execute on a single stream by default
 
//Creating a 3-channel Image container on GPU
CuviImage* gimg = new CuviImage(size,img->depth,3);
//Creating a sing channel Image container for output on GPU
CuviImage* gout = new CuviImage(size,img->depth,1);

//Uploading RGB image to GPU
gimg->upload(img->imageData,img->widthStep);

//function call
cuvi::colorOperations::RGB2Gray(gimg,gout);


Same example with Streams

Streams greatly improve performance of your application by hiding data processing time in data copying time. Instead of waiting for the complete image to be copied on GPU before processing, streaming enables processing the data as it arrives on GPU. Here's how you can use streaming in your application using CUVI:

//Number of data chunks
int streamCount = 1;

//Size of the image
CuviSize size = cuviSize(img->width,img->height);

//Creating a 3-channel Image container on GPU
CuviImage* gimg = new CuviImage(size,img->depth,3);
//Creating a sing channel Image container for output on GPU
CuviImage* gout = new CuviImage(size,img->depth,1);

//Height of each chunk in the image
size_t cHeight = gimg->height / streamCount;

//Creating streams
CuviStream **streams = new CuviStream*[streamCount];
for(int i=0; i<streamCount; i++)
     cuviCreateStream(&streams[i]);

 
//Chunks sizes for streams
//gpuChunk and hostChunk are mostly different because their pitch is different
size_t gpuChunk =  cHeight  * gimg->pitch,
       gpuOutChunk =  cHeight  * gout->pitch,
       hostChunk = cHeight  * host_img->widthStep;
       hostOutChunk = cHeight  * host_out->widthStep;

//Uploading image data to GPU in streams
for(int i=0; i<streamCount; i++)
       inputImage->upload(gimg->data + i*gpuChunk, host_img->imageData+ i*hostChunk, host_img->widthStep,streams[i]);
 
//Function call on each stream
for(int i=0; i<streamCount; i++)
       cuvi::colorOperations::RGB2Gray(gimg + i*gpuChunk, gout + i*gpuOutChunk, streams[i]);
 
//Downloading resultant data back to host in streams
for(int i=0; i<streamCount; i++)
       gout->download(host_out->imageData+ i*hostOutChunk , gout + i*gpuOutChunk, host_out->widthStep, streams[i]);
 

//Destroying streams
for(int i=0; i<streamCount; i++)
       cuviDestroyStream(&(streams[i]));
 
//Freeing GPU memory
delete gimg;
delete gout;
delete[] streams;