Get image from SurfaceTexture

vanKraster 181

Hi to all,

Does anyone know how to get image from SurfaceTexture ?

I have followed this example
https://github.com/xamarin/xamarin-forms-samples/tree/main/CustomRenderers/View

and inside
void TextureView.ISurfaceTextureListener.OnSurfaceTextureUpdated(SurfaceTexture surface)

I want to get the image from SurfaceTexture , is there possible ?

1 answer

Yonglun Liu (Shanghai Wicresoft Co,.Ltd.) 38,756 Reputation points Microsoft Vendor

2022-08-16T08:56:07.227+00:00

Hello,

No, there is no possible to get images from SurfaceTexture.

Referring to this official documentation SurfaceTexture, we could find the following description about SurfaceTexture:

Captures frames from an image stream as an OpenGL ES texture.

Therefore, you could not get the image from SurfaceTexture, due to it stores OpenGL ES texture.

Best Regards,

Alec Liu.

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
Please sign in to rate this answer.
Muhammad Umar Hassan 0 Reputation points

2024-06-25T14:18:49.5866667+00:00

I think a straight and forware "No" is unfair. It is possible to get an image from a surface texture. The technique requires rendering the texture on to a GL frame buffer. Here is an example that works for me, although I'm working on its optimization.
static byte[] getBitmap(SurfaceTexture surfaceTexture, int width, int height)

{

var st = Stopwatch.StartNew();

surfaceTexture.UpdateTexImage();

float[] transformMatrix = new float[16];

surfaceTexture.GetTransformMatrix(transformMatrix);

// Create framebuffer

int[] framebuffers = new int[1];

GLES20.GlGenFramebuffers(1, framebuffers, 0);

int framebuffer = framebuffers[0];

GLES20.GlBindFramebuffer(GLES20.GlFramebuffer, framebuffer);

// Create renderbuffer

int[] renderbuffers = new int[1];

GLES20.GlGenRenderbuffers(1, renderbuffers, 0);

int renderbuffer = renderbuffers[0];

GLES20.GlBindRenderbuffer(GLES20.GlRenderbuffer, renderbuffer);

GLES20.GlRenderbufferStorage(GLES20.GlRenderbuffer, GLES20.GlBlendSrcRgb, width, height);

GLES20.GlFramebufferRenderbuffer(GLES20.GlFramebuffer, GLES20.GlColorAttachment0, GLES20.GlRenderbuffer, renderbuffer);

// Attach texture to framebuffer

GLES20.GlFramebufferTexture2D(GLES20.GlFramebuffer, GLES20.GlColorAttachment0, GLES11Ext.GlTextureExternalOes, textures[0], 0);

// Check framebuffer status

int status = GLES20.GlCheckFramebufferStatus(GLES20.GlFramebuffer);

if (status != GLES20.GlFramebufferComplete)

{

Console.WriteLine("Framebuffer not complete: " + status);

}

// Read pixels

ByteBuffer buffer = ByteBuffer.AllocateDirect(width * height * 4);

buffer.Order(ByteOrder.NativeOrder());

GLES20.GlReadPixels(0, 0, width, height, GLES20.GlRgba, GLES20.GlUnsignedByte, buffer);

// Convert to BMP or PNG

byte[] pixels = new byte[width * height * 4];

buffer.Get(pixels);

Bitmap bitmap = Bitmap.CreateBitmap(width, height, Bitmap.Config.Argb8888);

bitmap.CopyPixelsFromBuffer(ByteBuffer.Wrap(pixels));

var bp19 = st.ElapsedMilliseconds; st.Restart();

byte[] imgData;

using (var stream = new MemoryStream())

{

bitmap.Compress(Bitmap.CompressFormat.Jpeg, 100, stream);

imgData = stream.ToArray();

// Use the PNG data

}

var bp18 = st.ElapsedMilliseconds; st.Restart();

// Cleanup

GLES20.GlDeleteFramebuffers(1, framebuffers, 0);

GLES20.GlDeleteRenderbuffers(1, renderbuffers, 0);

var bp20 = st.ElapsedMilliseconds; st.Restart();

return imgData;

}

It also requires that the surface texture be attached to a GL context.
static void createGLContext(SurfaceTexture surfaceTexture)

{

EGLDisplay display = EGL14.EglGetDisplay(EGL14.EglDefaultDisplay);

int[] version = new int[2];

EGL14.EglInitialize(display, version, 0, version, 1);

int[] configAttribs = {

EGL14.EglSurfaceType, EGL14.EglPbufferBit,

EGL14.EglRedSize, 8,

EGL14.EglGreenSize, 8,

EGL14.EglBlueSize, 8,

EGL14.EglAlphaSize, 8,

EGL14.EglRenderableType, EGL14.EglOpenglEs2Bit,

EGL14.EglNone

};

EGLConfig[] configs = new EGLConfig[1];

int[] numConfigs = new int[1];

EGL14.EglChooseConfig(display, configAttribs, 0, configs, 0, configs.Length, numConfigs, 0);

EGLConfig config = configs[0];

int[] contextAttribs = {

EGL14.EglContextClientVersion, 2,

EGL14.EglNone

}; EGLContext context = EGL14.EglCreateContext(display, config, EGL14.EglNoContext, contextAttribs, 0);

EGLSurface surface = EGL14.EglCreatePbufferSurface(display, config, new int[] {

EGL14.EglWidth, 1,

EGL14.EglHeight, 1,

EGL14.EglNone

}, 0);

EGL14.EglMakeCurrent(display, surface, surface, context);

// Create OpenGL texture

GLES20.GlGenTextures(1, textures, 0);

GLES20.GlBindTexture(GLES11Ext.GlTextureExternalOes, textures[0]);

surfaceTexture.AttachToGLContext(textures[0]);

}
Sign in to comment

Share via

Get image from SurfaceTexture

1 answer