亲宝软件园·资讯

展开

Flutter图像处理

handyTool 人气:2

背景

在之前自制的图像处理App中,使用了OpenGL处理图片,这次使用Metal替代OpenGL,来达到更好的性能,顺便熟悉一下Metal的渲染流程

基本思路

Flutter使用CVPixelBuffer和iOS交互,我们可以直接使用CVPixelBuffer创建MTLTexture,然后将MTLTexture设置为渲染目标。这样Metal框架可以直接将渲染结果写入CVPixelBuffer,达到更加高效的目的。

Metal环境设置

主要初始化DevicePipelineStateCommandQueue三个对象。我们需要依赖Device分配各种Metal资源,PipelineState管理着渲染流水线的各个环节的配置,比如vertex shader,fragment shader,输出像素格式等。CommandQueue用于管理执行的绘制命令。

_device = MTLCreateSystemDefaultDevice();
id<MTLLibrary> lib = [_device newDefaultLibrary];
id<MTLFunction> vertexFunc = [lib newFunctionWithName:vertexFuncName];
id<MTLFunction> fragFunc = [lib newFunctionWithName:fragFuncName];
MTLRenderPipelineDescriptor *renderPipelineDesc = [MTLRenderPipelineDescriptor new];
renderPipelineDesc.vertexFunction = vertexFunc;
renderPipelineDesc.fragmentFunction = fragFunc;
renderPipelineDesc.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm;
_pipelineState = [_device newRenderPipelineStateWithDescriptor:renderPipelineDesc error:nil];
_commandQueue = [_device newCommandQueue];

从CVPixelBuffer创建MTLTexture纹理

首先创建一个CVPixelBuffer对象

NSDictionary *pixelAttributes = @{( id )kCVPixelBufferIOSurfacePropertiesKey : @{}};
CVPixelBufferCreate(
            kCFAllocatorDefault,
                    imageWidth,
                    imageHeight,
            kCVPixelFormatType_32BGRA,
                    (__bridge CFDictionaryRef)pixelAttributes,
                    &_renderTargetPixelBuffer);

利用CVMetalTextureCacheCreateTextureFromImageCVPixelBuffer创建MTLTexture

CVReturn ret = CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, _mtContext.device, nil, &_textureCache);
CVMetalTextureRef renderTargetMetalTextureRef;
ret = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, _textureCache, _renderTargetPixelBuffer, nil, MTLPixelFormatBGRA8Unorm, imageWidth, imageHeight, 0, &renderTargetMetalTextureRef);
id<MTLTexture> mtlTexture = CVMetalTextureGetTexture(renderTargetMetalTextureRef);

渲染到纹理

CommandQueue获得一个CommandBuffer,用于保存需要执行的绘制命令

_activeCmdBuffer = [_commandQueue commandBuffer];

创建MTLRenderPassDescriptor设置本次绘制的相关配置,比如绘制到哪里,这里指定通过CVPixelBuffer创建出来的MTLTexture,是否清除当前内容,清除的颜色

MTLRenderPassDescriptor *renderPassDesc = [MTLRenderPassDescriptor new];
renderPassDesc.colorAttachments[0].texture = target;
renderPassDesc.colorAttachments[0].loadAction = MTLLoadActionClear;
renderPassDesc.colorAttachments[0].clearColor = MTLClearColorMake(0, 0, 0, 1);

通过CommandBufferMTLRenderPassDescriptor创建一个MTLRenderCommandEncoder

_activeEncoder = [_activeCmdBuffer renderCommandEncoderWithDescriptor:renderPassDesc];

指定MTLRenderCommandEncoder所在的PipelineState

[_activeEncoder setRenderPipelineState:_pipelineState];

使用MTLRenderCommandEncoder绑定BufferTexture,在Metal里,Uniform和Vertex Buffer 都是通过MTLBuffer绑定到Shader中

[_activeEncoder setVertexBuffer:vertexBuffer offset:0 atIndex:0];
[_activeEncoder setFragmentBuffer:uniformBuffer offset:0 atIndex:0];
[_activeEncoder setFragmentBuffer:texture offset:0 atIndex:0];

绘制图形

[_activeEncoder drawPrimitives:MTLPrimitiveTypeTriangle vertexStart:0 vertexCount:vertexCount instanceCount:1];

显式的结束MTLRenderCommandEncoder

[_activeEncoder endEncoding];

提交CommandBuffer

[_activeCmdBuffer commit];

等待绘制结束,如果你想要异步等待,需要在[_activeCmdBuffer commit]之前设置completedHandler

// 同步等待
[_activeCmdBuffer waitUntilCompleted];
// 异步等待
[_activeCmdBuffer addCompletedHandler:^(id<MTLCommandBuffer> _Nonnull buf) {
}];

到此绘制的内容就已经在CVPixelBuffer中了,再将CVPixelBuffer提交给Flutter显示即可。

加载全部内容

相关教程
猜你喜欢
用户评论