Speed up Android OpenGL ES glReadPixels by using PBO
[ ]Recently, I am working on a prototype which requires to use OpenGL to process the streams from Camere and save the processed results to a Video. Thus the overall pipeline is that:
CameraDevice -> ImageReader -> OpenGL -> MediaCodec
I choose OpenGL to proess the streams because the CPUs is too slow to perform real time processing. However, with OpenGL I find I still cannot achieve 30 FPS on 1080P, where the bottle neck nows comes from OpenGL -> MediaCodec.
To read the data from OpenGL, I use a texture, which is configured as:
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
//绑定纹理，这里的纹理是GL_TEXTURE_2D
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId);
GLES20.glUniform1i(mGLUniformTexture, 0);
Then read the results as:
GLES20.glReadPixels(0, 0, mRowStride, mInputHeight, GLES30.GL_RGBA, GLES30.GL_UNSIGNED_BYTE);
However the profiling indicates glReadPixels could take about 40ms on 1080P, which makes it impossible for 30FPS. The reason is that glReadPixels for texture is synchronous thus it needs to wait for all the OpenGL operations to finish before the reading.
After some research, I decided to use Pixel Pack Buffer (PBO), where glReadPixels for PBO is asynchronous while glReadPixels for texture is synchronous. To fully utilize this asynchronous, we will need to have two PBOs working in a ping-pong buffer: one for reading out and one for processing.
The code for configuring the PBO (note the configuration of texture is still required):
mPboIds = IntBuffer.allocate(2);
//生成2个PBO
GLES30.glGenBuffers(2, mPboIds);
//绑定到第一个PBO
GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, mPboIds.get(0));
//设置内存大小
GLES30.glBufferData(GLES30.GL_PIXEL_PACK_BUFFER, mPboSize, null,GLES30.GL_STATIC_READ);
//绑定到第而个PBO
GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, mPboIds.get(1));
//设置内存大小
GLES30.glBufferData(GLES30.GL_PIXEL_PACK_BUFFER, mPboSize, null,GLES30.GL_STATIC_READ);
//解除绑定PBO
GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, 0);
The code for reading data from PBO:
private ByteBuffer bindPixelBuffer() {
//绑定到第一个PBO
GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, mPboIds.get(mPboIndex));
ByteBuffer result = GLES30.glReadPixels(0, 0, mRowStride, mInputHeight, GLES30.GL_RGBA, GLES30.GL_UNSIGNED_BYTE);
//第一帧没有数据跳出
if (mInitRecord) {
unbindPixelBuffer();
mInitRecord = false;
return;
}
//绑定到第二个PBO
GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, mPboIds.get(mPboNewIndex));
//glMapBufferRange会等待DMA传输完成，所以需要交替使用pbo
//映射内存
ByteBuffer byteBuffer = (ByteBuffer) GLES30.glMapBufferRange(GLES30.GL_PIXEL_PACK_BUFFER, 0, mPboSize, GLES30.GL_MAP_READ_BIT);
//解除映射
GLES30.glUnmapBuffer(GLES30.GL_PIXEL_PACK_BUFFER);
unbindPixelBuffer();
//交给mRecordHelper录制
mRecordHelper.onRecord(byteBuffer, mInputWidth, mInputHeight, mRowStride, mLastTimestamp);
return result;
}
//解绑pbo
private void unbindPixelBuffer() {
//解除绑定PBO
GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, 0);
//交换索引
mPboIndex = (mPboIndex + 1) % 2;
mPboNewIndex = (mPboNewIndex + 1) % 2;
}
With PBO, the reading time is about 20ms.