Plan for new graphics rendering

This represents the plan for improved graphics rendering, utilizing arbitrarily transformed primitive rasterization. The goal here is to detail the new interfaces that each backend can implement.

Surface

A Surface is an array of "pixels" that is width x height in length. A Surface must have the following:

• refcount
• handle
• width
• height
• format
• usage
• data pointer

where refcount is a cumulative reference count on the surface, handle is a reference to a hardware implementation (if present), format is the data type of the pixel: either 8bit or 32bit; and usage can be one of the following:

• staging
• source
• render target

The source type is the typical usage where the Surface is used as the "texture" to sample in rendering. It can also be used as the source in a surface blit operation. Source surfaces are the only type that can directly manipulated at the pixel level. The render target type is for rendering transformed quads to it's surface, and being a destination target in a surface blit operation. By contrast, the normal source type cannot be the target of such operations (with the exception of a copy blit, with many restrictions). Render targets can also be used as a source texture in another operation.

Staging type is a software-only type. It can be 8bit or 32bit, and can be used as the source and destination of all of the surface operations. It can also work with source types in any operations where the source type is the source surface and staging type is the destination surface, or a valid function where source type may be a appropriate destination (such as gfx_surfaceCopy). Staging type surfaces cannot work directly with render targets. A source type surface must be an intermediary if data is to travel from render target to staging, or vice versa.

Why the distinction? Certain hardware acceleration techniques can optimize a certain resource type's usage. The software rasterizer does not take advantage of this case, however, so it will ignore the usage type.

The data pointer will contain a pointer to the raw data. If there is hardware acceleration, this will still be a valid array for source surfaces--however the hardware surface will not be updated without an appropriate call. Staging surfaces will never be hardware accelerated, and so this will always be a valid array upon creation.

Interfaces

Following is a list of suggested interfaces which can operate on surfaces of all usage types:

gfx_surfaceCreate( width, height, format, usage, pSurfaceOut )
gfx_surfaceDestroy( pSurfaceIn )
gfx_surfaceUpdate( pSurfaceIn, pRectDirtyRegion )
gfx_surfaceFill( color, pRect, pSurfaceIn )
gfx_surfaceStretch( pRectSrc, pSurfaceSrc, pPalette, bUseColorKey0, pRectDest, pSurfaceDest )
gfx_surfaceCopy( pRectSrc, pSurfaceSrc, pPalette, bUseColorKey0, pRectDest, pSurfaceDest )

gfx_surfaceCreate

This function will simply create a surface with the data values.

gfx_surfaceDestroy

This function will destroy a surface.

gfx_surfaceUpdate

This function is important only to hardware accelerated backends, as they are rendering from a separate, copied surface data. It's implementation in software is simply returning success. A typical scenario where this would be needed is if individual pixels are changed. Only source surfaces can be updated.

The optional pRectDirtyRegion specifies a rectangle of the surface to be updated, instead of the entire surface.

gfx_surfaceFill

The fill function clears a section of the surface to a color.

gfx_surfaceStretch

This function will stretch one surface on top of another, possibly scaling it horizontally and vertically. If the source format is 8bit and the destination format is 32bit, a passed in palette will be used to convert the formats. And if an 8bit source pixel has the value of 0 and bUseColorKey0 is true (non-zero), it will not be copied over. If one surface is a render target and another surface is staging types, the operation will fail because a source type surface must mediate the transfer.

gfx_surfaceCopy

This function will simply blit one surface onto another. If the source format is 8bit, and the destination is 32bit, a passed in palette will be used to convert the formats; and if the 8bit source pixel has the value of 0 and bUseColorKey0 is true (non-zero), it will not be copied over. If one surface is a render target and another surface is staging types, the operation will fail because a source type surface must mediate the transfer.

Palette's

A palette is an array of 256 numbers, each 32bits. They are used in rendering methods. They are defined as:

• handle
• palette array

where handle is a reference to a hardware implementation, if available.

Interfaces

The suggested interfaces for managing palettes include:

gfx_paletteCreate( pPaletteOut )
gfx_paletteDestroy( pPaletteIn )
gfx_paletteUpdate( pPaletteIn )

gfx_paletteCreate

This function simply creates a palette. In software, this is the equivalent of simply adding a structure containing a 256-entry array to an internal list, and returning a pointer to it.

gfx_paletteDestroy

This function just removes the palette from the list.

gfx_paletteUpdate

This function would typically be called when at least one entry is altered. This is important for hardware accelerated backends. In software rendering, it will simply return success.

Primitives

There are a few types of primitives that will be natively supported for rendering: quads, triangles, lines, and points. A quad, short for quadrilateral, is a four-sided, convex or concave polygon. It's structure consists of four vertices. A triangle is a three-sided, convex polygon. Because of the geometric nature of triangles, it is impossible for them to be concave. It consists of three vertices. A line is made of two vertices, and a point is a single vertex.

Each vertex may contain the following data (floats may eventually become fixed point ints):

• position (x,y) [2 floats]
• texture coordinates (u,v) [2 floats]
• vertex color [uint32]

There are 4 defined formats:

• VertexP (position)
• VertexPC (position, color)
• VertexPT (position, tex coordinates)
• VertexPTC (position, tex coordinates, color)

Polygons have to be rendered onto surfaces. Thus, surfaces created with the source usage are not valid targets. (Even though the software rasterizer can render to them, it shouldn't be done.) Only render target and staging type is valid for being rendered to. If staging type is a destination surface, the software rasterizer is invoked regardless of hardware acceleration.

Interfaces

The actual primitives' vertex data are managed by the engine. Because they typically have textures (or sampled surfaces) associated with them, and those surfaces may potentially be paletted, the interface is:

gfx_render( RenderParams* pParams )

where the RenderParams structure is defined as:

struct RenderParams
{
   union {
      VertexP* pVertexP;
      VertexPC* pVertexPC;
      VertexPT* pVertexPT;
      VertexPTC* pVertexPTC;
   };
   enum VertexFormat format;
   enum PrimitiveType type;
   Surface* pTexture;
   Palette* pPalette;
   int bUseColorKey0;
   uint32 argbModfier;
   SurfaceRect* pRectDest; [possibly embed the rect here]
   Surface* pSurfaceDest;
};

gfx_render

This function will be called whenever an arbitrarily transformed primitive must be rendered upon another surface.

The vertex pointers in the union are a pointer to the array of vertices to be rendered. vertexFmt is the vertex format, either P, PC, PT, or PTC. primitiveType is one of the primitive types, which will determine the number of vertices for processing. The next 3 parameters are only used if the vertex format specifies a texture coordinate. pTexture is a surface that is interpolated across the rasterizing primitive based on texture coordinates. pPalette is only used when the texture surface is 8bit. bUseColorKey0, if true (or non-zero), will cause the rasterizer to skip overwriting pixels on the destination surface if the index in the 8bit texture is 0. If the texture is 32bit, then this does not matter.

The argbModifier is a color that will either be applied to the whole object if the vertex format is strictly position, or will be applied to each vertex before rasterizing begins if the vertex format has colors, or will be applied during rasterization. If the value of argbModifier is 0xffffffff, it will not be applied to any rasterizing, save for the vertex format that is strictly position.

The pRectDest is a clipping region on the destination surface, and pSurfaceDest is a surface that is rendered to. This surface must be either a render target or staging type. If it is the render target type, it must be 32bit, and the texture may not be a staging surface. If it is the staging type, then the texture may not be a render target type. Finally, if the destination is the staging type, then it can be 8bit if the texture is 8bit.

Presenting the Backbuffer

The backbuffer is a special type of surface which can only be rendered/copied to, for it will be drawn on the screen. It cannot be the source surface in any function. When all rendering is complete, the backbuffer should be copied to from a render target and then presented. The backbuffer is never accessible to the engine.

Interfaces

There's only one interface associated with presenting the surface:

gfx_present( pSurfaceIn, pPalette )

gfx_present

This function accepts a render target or source surface as input. If it is in 8bit format, a passed in palette will be used to convert to 32bit when copying to the 32bit backbuffer.

Screen shots

Screen captures are impossible to obtain directly from render targets since it is inaccessible. In such a case, a solution is to copy a render target usage surface to a source usage surface, and then accessing the surface data from the source usage surface. For example:

Surface *pScreenShot, *pScene;
gfx_surfaceCreate( 320, 200, SF_32bit, SU_RenderTarget, &pScene );
gfx_surfaceCreate( 320, 200, SF_32bit, SU_Source, &pScreenShot );

RenderParams params;
params.pSurfaceDest = pScene;
...

gfx_render(&params);

gfx_present( pScene );

//when it's time to get a screen shot:
gfx_surfaceCopy( 0, pScene, 0, 0, 0, pScreenShot );

//access surface data
pScreenShot->pColorData[...]

Render Target and Staging

Render target and staging type surfaces cannot pass data to each other, but require an intermediary: a source type surface. The reason is that render target types, when implemented in hardware, are strictly in video memory, and staging types are strictly in software. An intermediate container is required for capturing the data from hardware. Source types have a container that is accessible both in hardware and software, so it makes a good candidate for receiving the render target data, then copying that to a staging surface area. The following is an example in implementing the copy:

Surface *pRenderTarget, *pIntermediate, *pStaging;

gfx_surfaceCreate( 50, 50, SF_32bit, SU_RenderTarget, &pRenderTarget );
gfx_surfaceCreate( 50, 50, SF_32bit, SU_Source, &pIntermediate );
gfx_surfaceCreate( 50, 50, SF_32bit, SU_Staging, &pStaging );

//render to the target
RenderParams params;
params.pSurfaceDest = pRenderTarget;
...

gfx_render(&params);

//copy first to intermediate surface, then staging area
gfx_surfaceCopy( 0, pRenderTarget, 0, 0, 0, pIntermediate );
gfx_surfaceCopy( 0, pIntermediate, 0, 0, 0, pStaging );

//access data
pStaging->pColorData[...]

Transferring data from staging surfaces to render targets may also be performed using a source surface as an intermediate medium.

Surface *pRenderTarget, *pIntermediate, *pStaging;

gfx_surfaceCreate( 50, 50, SF_32bit, SU_RenderTarget, &pRenderTarget );
gfx_surfaceCreate( 50, 50, SF_32bit, SU_Source, &pIntermediate );
gfx_surfaceCreate( 50, 50, SF_32bit, SU_Staging, &pStaging );

//write data to staging area
pStaging->pColorData[...]
...

//copy first to intermediate surface, then render target
gfx_surfaceCopy( 0, pStaging, 0, 0, 0, pIntermediate );
gfx_surfaceCopy( 0, pIntermediate, 0, 0, 0, pRenderTarget );

//use render target
...

In general, use of either of these methods is not efficient, nor necessarily useful. It is recommended to instead use staging and source type surfaces exclusively together, and when ready to use hardware acceleration, exclusively use source and render target type surfaces.

Error Codes

This section lists the error codes that this api may return. Following is a list of the error codes:

• GFX_SUCCESS (0)
• GFX_INVALID_POINTER (-1)
• GFX_INVALID_PARAMETER (-2)
• GFX_NOT_SUPPORTED (-3)
• GFX_NOT_READY (-4)
• GFX_OUT_OF_MEMORY (-5)

GFX_SUCCESS

This error code will return upon successful operation.

GFX_INVALID_POINTER

This error returns when a pointer passed in is invalid (ie. null pointer). This can happen when a surface or palette is being created and the destination pointer is invalid.

GFX_INVALID_PARAMETER

This error returns when a parameter is not logical. For example, creating a surface that is 0 pixels wide.

GFX_NOT_SUPPORTED

This error returns when a certain operation is not supported. This can happen when a staging surface is attempted to be copied to a render target surface, or a 32bit surface is trying to copy to an 8bit surface.

GFX_NOT_READY

This error returns when a function call is not ready. This will commonly occur if a backend has lost the device context and, for example, is unable to present surfaces to the backbuffer.

GFX_OUT_OF_MEMORY

This error returns when the system is out of memory.