Using Streams with CUVI
CUVI framework provides a way to use streams with minimal coding effort. Each function call in CUVI takes an optional parameter to specify the stream on which it should run. The code below shows how a simple function call of CUVI can be divided into streaming calls on GPU. For most of the cases this will result in better performance as copying image data to GPU and processing that data on GPU happens simultaneously.
CUVI example
In this example we use CUVI's RGB2Gray function from Color Operations module on a full HD input image
//You may notice that all the functions below take an optional parameter for stream
//If the users doesn't wish to use it the program will execute on a single stream by default
//Creating a 3-channel Image container on GPU
CuviImage* gimg = new CuviImage(size,img->depth,3);
//Creating a sing channel Image container for output on GPU
CuviImage* gout = new CuviImage(size,img->depth,1);
//Uploading RGB image to GPU
gimg->upload(host_img->imageData, host_img->widthStep);
//function call
cuvi::colorOperations::RGB2Gray(gimg,gout);
//Download resultant gray image back to host
gout->download(host_out->imageData, host_out->widthStep);
|
Same example with Streams
Streams greatly improve performance of your application by hiding data processing time in data copying time. Instead of waiting for the complete image to be copied on GPU before processing, streaming enables processing the data as it arrives on GPU. Here's how you can use streaming in your application using CUVI:
//Number of data chunks
int streamCount = 4;
//Size of each image chunk
CuviSize size = cuviSize(host_img->width, host_img->height/streamCount);
//Creating 3-channel Image containers on GPU for each chunk
CuviImage ** gimg = new CuviImage*[streamCount];
for(int i=0; i<streamCount; i++)
gimg[i] = new CuviImage(size,img->depth,3);
//Creating single channel Image containers for output on GPU for each chunk
CuviImage ** gout = new CuviImage*[streamCount];
for(int i=0; i<streamCount; i++)
gout[i] = new CuviImage(size,out->depth,1);
//Creating a stream against each chunk of data
CuviStream **streams = new CuviStream*[streamCount];
for(int i=0; i<streamCount; i++)
cuviCreateStream(&streams[i]);
//Uploading image data to GPU in streams
for(int i=0; i<streamCount; i++)
gimg[i]->upload(host_img->imageData + i*(size.height*host_img->widthStep), host_img->widthStep, streams[i]);
//Function call on each stream
for(int i=0; i<streamCount; i++){
cuvi::colorOperations::RGB2Gray(gimg[i],gout[i],streams[i]);
//Downloading resultant data back to host image in streams
for(int i=0; i<streamCount; i++)
gout[i]->download(host_out->imageData + i*(size.height * host_out->widthStep), host_out->widthStep, streams[i]);
//Don't forget to destroy streams and free memory
|
Multi-GPU in CUVI
Applications that use CUVI also have the liberty to scale up on a Multi-GPU environment without changing a single line of code. We now know how to play with streams in CUVI. Multi-GPU is nothing more than just dividing those chunk's execution across all the GPU devices installed in the machine. You can write a single piece of code with few checks and error handling that will run of a single GPU machine as well as on Multi-GPU machine while using full capability of that machine.
cuviGetDeviceCount(&DeviceCount);
if(DeviceCount>1){
//Multi-GPU code here
}
|
If you have more than one GPU installed on the machine you can divide stream execution among them by just selecting the device before any CUVI call using cuviSetCurrentDevice(X); where X is the device id and its range is {0,N-1} in a machine containing N CUDA capable GPUs. Any function call of CUVI following cuviSetCurrentDevice(X) will execute on the device X until you set the current device to another.