Frame Capture Time Increasing

Hi,
I am using the RB5 Vision Kit. I have been stuck on some points. I am trying to get the video from the main camera with OpenCV.(You can see my codes below). The problem is read time of a frame increases after some time.
For example, Initially, each frame is reading something like 0.7-0.9 milliseconds but after some time, 20-25 seconds later, It’s starting to slow down, ie. average read time increasing sharply, even If I wait for enough, ie. 2 min, read time going to 30 milliseconds.
What’s the problem with that?
Thanks.

cap.open(“qtiqmmfsrc camera=0 name=qmmf ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! videoconvert ! appsink”, cv::CAP_GSTREAMER);

tm.start();
cap.read(frame);
tm.stop();
qDebug() << "Average time per iteration in miliseconds: " << tm.getAvgTimeMilli() << captureFrameIndex_u64;